Designing Cloud‑Native Distributed Database Architecture: Lessons from TiDB

In this talk, senior PingCAP expert Wang Pengfei shares a step‑by‑step analysis of cloud‑native distributed database design, using TiDB as a concrete example.

01 Cloud‑Native Understanding – Cloud‑native is a means to achieve elastic, pay‑as‑you‑go resources while meeting diverse SLA requirements; a cloud‑native database must be distributed and leverage cloud infrastructure to reduce total cost.

02 Architecture Design Thinking – TiDB’s current IDC‑centric design separates compute (TiDB) and storage (TiKV) but still assumes on‑premise assumptions. By moving the compute‑storage boundary further to the persistent storage layer (e.g., S3), both TiDB and TiKV become stateless services. This enables the use of cloud block storage (EBS) and object storage (S3) to achieve lower cost, cross‑AZ durability, and flexible pricing.

The proposed architecture stores data in S3 with an asynchronous cache layer; writes go through a write‑ahead log (WAL) that is replicated across AZs. Reads first hit the cache, falling back to S3 only when necessary. Transaction logs are synchronously persisted to guarantee durability, while the cache provides high‑performance access.

03 Escape, Disaster Recovery, and Scaling – Multi‑AZ disaster recovery is simplified because S3 is inherently cross‑AZ; the new design reduces the number of required nodes for HA. Scaling becomes easier since new nodes load data from S3 asynchronously without costly data re‑balancing, and expansion can be performed without impacting existing workloads.

The design also discusses trade‑offs such as cache‑hit‑rate guarantees, storage format choices (KV vs. columnar), and the potential to replace S3 with alternatives like HDFS or EC‑enabled storage for further cost savings.

Overall, the architecture aims to lower operational costs, improve elasticity, and keep pace with rapid cloud storage innovations while maintaining the reliability required for OLTP workloads.