Tencent SuperSQL: A Unified Adaptive Big Data Computing Platform

In the era of big‑data democratization, organizations face fragmented data islands caused by heterogeneous sources, inconsistent SQL interfaces, and complex engine-specific syntax, leading to costly manual migrations and sub‑optimal performance.

SuperSQL addresses these issues with a four‑layer architecture—core (metadata, CBO, RBO, HBO), compute (integration of Presto, Spark, StarRocks, etc.), resource (cloud‑on‑premise elasticity), and storage (DOP‑based HDFS‑compatible interface)—providing a unified SQL entry that automatically selects the most efficient engine.

The platform’s key techniques include plugin‑based multi‑engine SQL parsing, federated push‑down of complex operators, incremental and sampled statistics for cost‑based optimization, history‑based workload analysis (HBO) for engine selection, adaptive materialized views, and centralized data‑masking for security, all orchestrated through JDBC, CLI, SDK, Python client and RESTful APIs.

Since deployment, SuperSQL has achieved 30‑70% reduction in failed SQL jobs, up to 88% failure avoidance with HBO‑ML, massive resource savings (tens of TB memory, hundreds of CPU cores), and significant community impact with open‑source contributions, standard‑setting participation and a growing user base across private and public clouds.

The Q&A section clarifies open‑source plans, engine support (Spark, Presto, StarRocks, potential Doris and Flink), syntax conversion via Apache Calcite, security handling, cross‑region considerations, and future roadmap for execution‑plan integration.