Tencent SuperSQL Sets New TPC‑DS World Record, Leads in Performance and Cost Efficiency

TPC‑DS Benchmark Highlights

Recently the Transaction Processing Performance Council (TPC) published the results of Tencent’s SuperSQL on the TPC‑DS 100 TB benchmark. SuperSQL achieved a composite performance score of 654 million, nearly ten times the previous record, while its performance‑per‑dollar cost dropped to 11.04 CNY, about one‑sixth of earlier results, making it the global leader in both speed and cost‑effectiveness.

Why TPC‑DS Matters

TPC‑DS is the de‑facto “gold standard” for evaluating data‑analysis systems at scale. It measures a system’s ability to load data, execute complex queries, handle concurrent workloads, and maintain data, providing a reproducible, audited comparison across vendors.

Challenges of the 100 TB Scale

Massive data volume : billions of rows stress compute, storage, network and memory.

Complex data distribution : skew can amplify “long‑tail” effects.

Full‑stack workload : the test covers loading, query throughput, and data maintenance.

Strict audit : configuration, execution, performance and cost must meet TPC rules.

Performance Gains in Core Phases

Data loading : 449.4 s, 7.6× faster than the previous best (3427.1 s).

Serial queries : 177.5 s, 16.5× speed‑up.

Parallel queries : 898.8 s total, 11.7× faster.

Data updates : 196.9 s total, 6.7× faster.

Key Architectural Innovations

SuperSQL is built on the self‑developed TEngine vectorized compute layer, which unifies stream‑batch execution and supports both CPU and GPU. Its modular “Lego‑style” design provides:

Query optimizer : global‑view optimization with “once‑optimized, multi‑mode” reuse.

Distributed scheduler : flexible support for heterogeneous resources.

Universal compute base : a single execution layer that benefits all downstream engines.

Heterogeneous acceleration : native CPU‑GPU collaboration.

Deep Dive 1: Advanced Query Optimization

Traditional optimizers combine rule‑based (RBO) and cost‑based (CBO) techniques, but they struggle with repeated sub‑computations, runtime filter cost, and skewed statistics. SuperSQL introduces:

AutoCTE : automatically extracts common sub‑plans across query branches and materializes them when beneficial.

Runtime‑filter cost modeling : incorporates filter generation, transmission and application costs into the global cost model.

History‑Based Optimization (HBO) and Adaptive Query Execution (AQE) : uses real‑time execution statistics to correct static estimates and adjust plans on the fly.

Deep Dive 2: TEngine Vectorized Engine

To meet the 100 TB pressure test, TEngine addresses three classic MPP bottlenecks:

Network storm : massive node‑to‑node traffic can cause bandwidth congestion.

Pipeline stalls : operator waiting blocks CPU cycles.

Cluster long‑tail : slow nodes drag overall progress.

Solutions include:

Adaptive encoding : runtime‑aware dict, delta, RLE compression based on operator blocking points.

Micro‑adaptive execution : per‑operator and per‑batch tuning for grouping, joins, window functions.

Unified memory manager : custom allocator, hierarchical quota tracking, and deterministic lifecycle reclamation.

Adaptive shuffle : dynamic partition concurrency scaling and a vRDMA‑based zero‑copy shuffle path with custom congestion control.

Deep Dive 3: Large‑Model‑Driven Intelligent Diagnosis

SuperSQL integrates a large‑model‑powered “DataAgent” that closes the loop from problem discovery to root‑cause analysis and strategy recommendation. Highlights:

Platform performance diagnosis : agent‑level observability identified >20 operator‑level bottlenecks, reducing end‑to‑end latency by 20.9 % and CPU core‑time by 46 %.

Workload anomaly attribution : >90 % correct diagnosis of abnormal workloads, 69.2 % accurate ticket resolution.

Data acceleration & parameter recommendation : optimized Belady‑style pre‑loading cut slow SQL count by 25 % and P50 latency by 27 %; Bayesian black‑box tuning cut SQL latency by 32 % and CPU time by 66 %.

DataAgent Achievement

In the enterprise‑grade Text‑to‑SQL Spider 2.0‑Lite leaderboard, DataAgent achieved 76.23 % accuracy, surpassing solutions from Snowflake, Oracle and other vendors.

Business Impact

The benchmark breakthrough translates into three tangible benefits for data platforms: higher query throughput, lower cost per query, and stronger scalability for PB‑level workloads.

Conclusion and Outlook

The TPC‑DS 100 TB result validates SuperSQL’s end‑to‑end capabilities—from optimizer and vectorized execution to memory management, adaptive shuffle, and AI‑driven diagnostics. The team plans to continue evolving these technologies to serve real‑world business needs.