ByteHouse Cloud‑Native Data Warehouse Performance Whitepaper: Architecture, Optimizations, and Benchmark Results

Background

ByteHouse is ByteDance’s cloud‑native data warehouse built on a re‑engineered ClickHouse core, offering storage‑compute separation, multi‑tenant management, and native cloud deployment, with significant gains in scalability, stability, operability, performance, and resource utilization.

Performance Optimizations – Complex Queries

Rule‑Based Optimizer (RBO)

RBO applies heuristics such as column pruning, partition pruning, expression simplification, sub‑query decorrelation, predicate push‑down, redundant operator elimination, outer‑join to inner‑join conversion, operator push‑down to storage, and distributed operator splitting.

Full decorrelation enables all TPC‑DS queries to run.

Non‑equi joins are evaluated directly in the join operator, doubling performance.

Count‑distinct on multiple columns is accelerated via replication‑based parallelism.

Cost‑Based Optimizer (CBO)

CBO uses a cascade search framework to generate physical plans while minimizing cost based on statistics; join order enumeration is accelerated with join‑graph partitioning, and CTEs support inline, shared, and partial inline costing.

Distributed Plan Generation

The optimizer merges single‑node and distributed planning phases, generating global‑cost optimal plans and reducing shuffle by leveraging metadata‑driven data distribution; a custom exchange module provides two‑layer data transfer and operator layers.

Runtime Filters

Dynamic runtime filters built during hash‑join probe prune irrelevant rows early, reducing data movement and computation, with support for distributed, local, and shuffle‑aware filters.

Parallelism Refactoring

Adaptive aggregation streaming and bucket‑table awareness cut unnecessary intermediate merges and shuffle, boosting parallel execution.

Performance Optimizations – Wide‑Table Queries

ByteHouse introduces global dictionary encoding, zero‑copy data handling, and an uncompress cache to accelerate wide‑table workloads.

High‑Concurrency Point‑Query Optimizations

Short‑circuit execution plans collapse the two‑stage plan into a single segment for point‑lookups, unique‑table point‑query indexes provide O(1) row location, and lightweight partition pruning with min‑max mark indexes limits data reads; bucket cache and row‑store cache further reduce lock contention.

Performance Results

On 100 GB benchmarks, ByteHouse outperforms a leading open‑source OLAP engine by 6‑15× on TPC‑DS, over 100× on TPC‑H, and 3.6× on SSB wide‑table queries, with additional gains from the described optimizations.

Conclusion

Through extensive architectural and runtime enhancements, ByteHouse delivers high‑performance real‑time analytics across eight key scenarios, including crowd selection with a custom BitEngine that yields 10‑50× speedups, while maintaining cloud‑native operability and ecosystem compatibility.