Overview of Tencent TBase: Distributed HTAP Database Architecture and Capabilities

TBase is an enterprise‑grade distributed HTAP (Hybrid Transaction/Analytical Processing) database developed by Tencent's TEG data platform team based on open‑source PostgreSQL. It provides strong consistency, high‑concurrency online transaction processing (OLTP) and high‑performance online analytical processing (OLAP) within the same cluster.

1. Development History – TBase started internal trials in 2009, initially supplementing TDW with PostgreSQL for small‑scale analytics. As business grew, a more efficient transaction engine was needed, leading to continuous investment. The first version was deployed internally in 2014, followed by a large‑scale WeChat Pay deployment in 2015 (over 600 million transactions per day). Subsequent versions (V2 in 2018, V2.1.0 in 2023) added transaction and query optimizations, and the project was open‑sourced in November 2019.

2. PostgreSQL Background – PostgreSQL is an open‑source RDBMS with a permissive BSB‑style license, actively maintained (currently at version 12). TBase inherits PostgreSQL’s robust relational capabilities and extends them with distributed transaction support, multi‑active data, and enterprise‑grade security (three‑level separation, data masking, encryption).

3. HTAP Positioning and Architecture – TBase targets three workload categories: OLAP (large data, cost‑sensitive), OLTP (high‑throughput, low‑latency transactions), and HTAP (combined OLTP/OLAP in a single system). The cluster consists of GTM (Global Transaction Manager), Coordinators (CN) and Datanodes (DN). GTM provides global timestamp (GTS) and transaction coordination, eliminating the single‑point bottleneck of traditional GTM XID handling.

4. Distributed Transaction Mechanism – TBase uses a two‑phase commit (2PC) protocol enhanced with GTS‑based MVCC visibility. GTM distributes monotonic timestamps, and each DN records GTS in a dedicated store file, allowing transaction commit without per‑tuple locking. This design achieves up to 12 million TPS on a single server.

5. Data Distribution Strategies – Three distribution models are supported: replication (full copy on each node, suitable for small tables), hash distribution (data partitioned by hash of a distribution key), and range distribution (data split by value ranges). TBase primarily uses replication and enhanced hash distribution.

6. Query Optimization – Both rule‑based optimization (RBO) and cost‑based optimization (CBO) are employed, with a focus on CBO for complex workloads. Parallelism is applied at node, process, and instruction levels. HashJoin and aggregation operators have been parallelized, and redistribution phases now support parallel execution.

7. Enterprise‑Level Features – TBase offers multi‑active, multi‑center deployment, fine‑grained security (row‑level, column‑level, encryption, masking), hot‑cold data separation, and seamless horizontal scaling with minimal service disruption. The open‑source version (based on PostgreSQL 10) continues to track upstream features.

8. Frequently Asked Questions (selected) – Deployment can start with a single DN, CN, and GTM; larger clusters use multiple DNs for better distribution. DN nodes use primary‑secondary replication rather than Raft. Distributed transactions follow an enhanced 2PC with GTM‑coordinated timestamps. Storage is currently row‑oriented with future plans for columnar support.