Evolution and Architecture of Distributed Databases: A TDSQL Case Study

This article is a written summary of Li Haixiang’s talk at the Tech TVP Developer Summit titled “The Ice and Fire of Data – From Online Database Technology to Massive Data Analytics”. It reviews the evolution, architecture, and cutting‑edge techniques of distributed databases, with a focus on Tencent’s TDSQL.

1. Distributed Database Architecture

The speaker outlines three main parts of the presentation: the historical evolution of distributed databases, core technical concepts, and Tencent’s recent work on TDSQL. TDSQL is an HTAP‑oriented distributed database that emphasizes strong consistency. The concept of a “full‑time database” (HTAC/HTAP) was introduced in 2017‑2018, and the product has since been refined for strong‑consistency guarantees.

2. Classic Distributed System Architectures

The talk reviews classic architectural models such as Shared‑Nothing, Shared‑Memory, Shared‑Disk, and Shared‑Everything, emphasizing how hardware advances drive software design. The goal is to decouple tightly coupled modules so that components can be assembled like building blocks, improving extensibility.

3. Classic Database Technologies

The speaker traces key milestones: the relational model (Codd, 1970), the invention of SQL and transaction processing (1974), the emergence of ER modeling, cost‑based query optimization (1980), the volcano execution model, and the split of OLTP/OLAP in 1993. Gartner’s HTAP concept (2014) and Google Spanner (2012) are highlighted as turning points toward New‑SQL systems.

4. Distributed Transactions and Consistency

Transaction processing is identified as the core of distributed databases. The speaker explains data anomalies that arise from concurrent reads/writes, the ACID properties (especially Consistency and Isolation), and the classification of anomalies into multiple isolation levels (SQL standard, James Gray’s eight levels, etc.).

The speaker notes that TDSQL has built a systematic taxonomy of data anomalies and open‑sourced the research as the 3TS (Tencent Transaction Processing Testbed System). Various concurrency control algorithms (locking, timestamp, optimistic) are discussed.

5. Challenges in Distributed Consistency

Beyond classic transaction anomalies, distributed systems must also satisfy distributed consistency models (linearizability, sequential consistency, causal consistency, etc.). The speaker points out that combining these models with ACID leads to a combinatorial explosion of possible guarantees, many of which lack mature theory or implementations.

Google Spanner achieves strong consistency but suffers from performance limitations, especially for latency‑sensitive workloads such as finance. TDSQL aims to provide comparable strong consistency with higher performance.

6. HTAP‑Based Strong Consistency in TDSQL

TDSQL’s HTAC architecture integrates HTAP capabilities with multi‑level serializability. Experiments on Google Cloud Spanner, Greenplum, and an MIT open‑source system show that TDSQL outperforms these baselines.

7. Summary

The speaker concludes that the evolution of distributed databases is driven by internal requirements (high reliability, availability, performance, ease‑of‑use) and external pressures (massive data volumes, AI workloads, new hardware such as persistent memory and RDMA). Decentralization—treating all nodes as peers—is emphasized as a fundamental design principle.

For deeper technical details, the audience is invited to explore the 3TS system, which investigates transaction processing, performance, security, and the classification of data anomalies.