Inside TBase: Architecture, Distributed Transactions, and Cloud‑Native Security

Database Product Classification

Databases can be classified by business scenario (OLAP, OLTP, HTAP), by generation (Old SQL, NoSQL, New SQL), and by architecture (monolithic, shared‑storage, MPP). Representative products include ClickHouse (OLAP), MySQL (OLTP), TiDB (HTAP) and TBase (MPP).

Physical Architecture of TBase

TBase is built on PostgreSQL and follows a three‑layer physical design:

Global Transaction Manager (GTM) : maintains a global transaction list and provides snapshot information to all nodes.

Coordinator (CN) : the SQL entry point; multiple CNs are peers and guarantee consistent results.

Data‑Bus Layer : connects the coordinators with the data nodes.

Data Nodes (DN) : each DN stores a local shard and its own metadata.

The architecture enables global transaction consistency, resource isolation, and independent scaling of compute and storage.

Distributed Transaction Model

TBase implements distributed snapshot isolation (MVCC) using GTM snapshots. Each transaction obtains a snapshot of the active transaction list, guaranteeing atomicity across shards. The transaction list size grows O(N) with concurrent transactions, and snapshot retrieval incurs O(N²) network traffic, creating a scalability bottleneck.

Two concurrency‑control families are offered:

Physical‑time based (Google Spanner style): relies on a globally synchronized physical clock (GPS + atomic clocks) and incurs a minimum latency of ~6 ms per transaction.

Logical‑time based (TSO/GTS cluster): uses a monotonically increasing logical timestamp generated by a dedicated cluster.

Logical Clock (GTS) and Performance

TBase’s GTS (Global Timestamp Service) starts at zero and increments unidirectionally. The GTS state is replicated via streaming to ensure reliability. On a 24‑core server a single GTS cluster can sustain up to 12 million QPS, providing the throughput required for high‑volume workloads.

Storage Layout: Row vs. Column

Row storage writes complete rows sequentially, which is optimal for OLTP workloads where a transaction typically accesses many columns of a single row.

Column storage stores values of the same column together, reducing I/O for analytical aggregations and improving OLAP query performance. TBase can use either layout depending on the workload.

Data Distribution Strategies

TBase supports three sharding methods:

Replicated tables : a full copy on every DN, suitable for small or read‑heavy tables.

Hash‑based sharding : rows are distributed by a hash of the distribution key; may suffer from data skew.

Range‑based sharding : rows are placed according to a key range; useful for time‑series or warehouse scenarios.

In practice TBase mainly uses replicated and hash distributions, with range sharding applied in specific data‑warehouse cases.

Distributed Query Execution

Two execution strategies are available:

PUSH QUERY : the SQL is shipped to DN nodes, where filtering and aggregation occur before results are sent to the coordinator. This reduces data transfer for large tables.

PULL DATA : raw data is pulled to the coordinator for processing, which can be faster for small tables.

The optimizer also decides between SQL shipping (push full SQL to DN) and PLAN shipping (generate the execution plan on CN and push the plan to DN), balancing plan‑generation overhead against execution efficiency.

Query Optimizer

TBase includes both a rule‑based optimizer (RBO) and a cost‑based optimizer (CBO). RBO quickly produces plans based on known query patterns, while CBO evaluates multiple alternatives using table statistics and chooses the lowest‑cost plan, which is essential for complex joins and large‑scale queries.

Multiple‑Layer Security (MLS)

MLS separates three administrative roles:

Security administrator : defines security rules.

Audit administrator : defines audit policies.

Data administrator (DBA): manages data objects.

Security mechanisms include:

Row‑level security based on level , catalog , and group attributes.

Column‑level ACLs that specify which principals may read or write each column.

Transparent at‑rest encryption and on‑the‑fly data masking for unauthorized users.

Backup and Recovery

Cold backup is performed per DN shard. Each DN writes its shard to an external storage backend (disk array, HDFS, object store) together with a logical timestamp. The timestamps guarantee a consistent recovery point across the cluster, allowing point‑in‑time restoration without a global lock.

Open‑Source Availability

The TBase source code is publicly available at https://github.com/Tencent/TBase.

Key Q&A Highlights

Distributed table backup requires each DN to back up its shard while the backup coordinator enforces cluster‑wide consistency.

Logical timestamps are globally monotonic, eliminating clock drift and providing a stable identifier for recovery.

Security controls are exposed via APIs; rules are created by the security administrator and enforced using token‑based access checks.

Hot‑cold data separation is defined manually (e.g., data older than four months is cold) but the migration between SSD (hot) and HDD (cold) is automated by the control system.

PUSH QUERY is preferred for large data volumes; PULL DATA is more efficient for small tables.