Deploying a TiDB Cluster with Ansible: A Complete Step‑by‑Step Guide

TiDB Overview

TiDB clusters consist of three core components: TiDB Server, Placement Driver (PD) Server, and TiKV Server. Additional components include TiSpark for OLAP workloads and TiDB Operator for cloud‑native Kubernetes deployments.

TiDB Server handles SQL requests, is stateless, and can be scaled horizontally behind a load balancer.

Placement Driver (PD) stores metadata, performs scheduling and load balancing, and generates globally unique transaction IDs. PD uses the Raft protocol for high availability; an odd number of PD nodes is recommended.

TiKV Server provides a distributed transactional key‑value store. Data is stored in Regions, each covering a key range, and replicated via Raft groups managed by PD.

TiSpark enables Spark SQL to run directly on TiDB storage, combining OLTP and OLAP capabilities.

TiDB Operator simplifies deployment and management of TiDB clusters on Kubernetes, offering one‑click deployment, multi‑cluster support, automatic operations, and self‑healing.

Preparation

A topology with separate nodes for TiDB, PD, and TiKV is required. A control machine runs Ansible to execute commands on the other nodes.

qabb-qa-tibd0/10.215.20.24

Control / PD / Monitoring

qabb-qa-tibd1/10.215.20.19

TiKV

qabb-qa-tibd2/10.215.20.7

TiKV

qabb-qa-tibd3/10.215.20.13

TiKV

The control node uses Ansible to perform all deployment actions.

Deployment Steps

3.1 Obtain the Latest Version

git clone https://github.com/pingcap/tidb-ansible.git

Install Ansible and its dependencies:

cd /data/users/yangqilong/tidb-ansible && \
sudo pip install -r ./requirements.txt && \
ansible --version

3.2 Verify User Permissions

ansible -i inventory.ini all -m shell -a 'whoami'

All nodes should return the deployment user (e.g., yangqilong). Using -n returns root.

3.3 Download TiDB Binaries (Online)

ansible-playbook blocal_prepare.yml

3.4 Initialize System Environment

ansible-playbook bootstrap.yaml

Common issues include insufficient ulimit -n values, failing fio benchmarks, and missing ext4 mount options. Adjust /etc/security/limits.conf, remove unnecessary benchmark tasks, and modify mount parameters as needed.

3.5 Deploy TiDB Cluster

ansible-playbook deploy.yml

Successful deployment shows a confirmation screen:

If the blackbox_exporter binary fails, edit

tidb-ansible/roles/blackbox_exporter/tasks/binary_deployment.yml

and comment out the setcap task:

#- name: blackbox_exporter binary add CAP_NET_RAW capability
#  command: setcap cap_net_raw+ep "{{ deploy_dir }}/bin/blackbox_exporter"
#  become: true

3.6 Start the Cluster

ansible-playbook start.yml

After health checks, the cluster starts and can be accessed with a MySQL client:

Conclusion

The first full installation of a TiDB cluster is complex, similar to installing Oracle RAC, requiring careful handling of user permissions, system parameters, and package checks. Teams with independent operations can package and deploy the cluster themselves, then explore TiDB's features and operational best practices.