TiDB Architecture, Design, and Real‑World Application

TiDB is a widely used distributed NewSQL database developed in China. This presentation covers the evolution of database technology, TiDB’s design principles, core components, ecosystem tools, HTAP capabilities, and its deployment at DeWu.

Database Evolution

Before 2008, single‑node relational databases (e.g., Oracle) dominated but suffered from high cost and poor linear scalability. From 2008 to 2013, the explosion of data led to the rise of distributed and NoSQL systems (MongoDB, HBase), which lacked strong transaction support. Since 2013, NewSQL databases such as Google Spanner, CockroachDB, and TiDB have combined NoSQL scalability with ACID guarantees.

TiDB Design and Architecture

TiDB follows a pure distributed architecture with excellent elasticity. It exposes the MySQL protocol, supports SQL, and provides built‑in high availability.

Key Components

TiDB Server : Stateless SQL layer that parses queries, generates distributed execution plans, and forwards data requests to storage nodes.

PD Server : Metadata manager (the “brain”) that stores topology, schedules regions, and assigns transaction IDs. It runs as a quorum of at least three nodes.

TiKV Server : Distributed transactional key‑value store. Data is stored in Regions, each replicated (default three copies) for high availability. TiKV provides snapshot isolation and participates in Raft‑based replication.

Automatic Scaling

The expansion process moves hot Regions from overloaded nodes to newly added nodes, updates leaders via PD, and creates new replicas, ensuring seamless capacity growth.

Multi‑Data‑Center Deployment

TiDB supports a two‑region, three‑center architecture (production, same‑city disaster‑recovery, remote disaster‑recovery) using Raft to maintain consistency while offering high availability across cities.

SQL‑to‑KV Mapping

Each row becomes a KV pair with a TableID‑based prefix; indexes become KV pairs with TableID+IndexID prefixes. This layout can cause hotspot regions when writes concentrate on sequential keys.

Secondary Indexes

TiDB provides global secondary indexes. While they accelerate queries, excessive indexes increase write latency, storage cost, and optimizer overhead. Proper index selection (high‑cardinality columns, covering indexes) is essential.

Distributed Transactions

TiDB implements a Percolator‑style two‑phase commit (2PC). The pre‑write phase locks keys, and the commit phase finalizes the transaction after the primary key is committed.

TiDB Ecosystem Tools

• DM : Data Migration from MySQL/MariaDB to TiDB (full and incremental). • BR : Backup & Restore for large‑scale clusters. • TiDB Binlog & TiCDC : Real‑time change data capture and incremental sync. • Monitoring : Prometheus + Grafana + Alertmanager.

HTAP and TiFlash

TiFlash is TiKV’s column‑store extension, enabling hybrid transactional/analytical processing (HTAP). It replicates data asynchronously via Raft Learner, provides snapshot isolation, and can be automatically selected by TiDB for queries that benefit from columnar storage.

Application at DeWu

DeWu runs four TiDB clusters (≈60 nodes, 27 TB data). TiDB is used for high‑throughput, linearly scalable workloads where single‑node MySQL would be insufficient. Scenarios with modest data volume or low latency requirements still favor traditional MySQL.

Conclusion

The talk highlighted TiDB’s architecture, scaling mechanisms, ecosystem, and practical usage. Future sessions may explore scenario‑specific best practices.