How ByteDance Scaled to 10 EB: Evolution of Its Cloud‑Native Database Architecture

Evolution of ByteDance Databases

Database technology has always been a crucial part of IT, and in the cloud‑native era, integration of cloud infrastructure and databases has addressed traditional pain points, offering high scalability, full automation, rapid deployment, cost savings, and easy management.

From 2018 to 2021, as business and data grew rapidly, ByteDance's distributed database system made remarkable progress. The number of application‑side containers grew from 50,000 to 7.5 million, surpassing 10 million, forming a solid cloud‑native foundation that supports the entire business ecosystem.

These 10 million containers host over 100,000 micro‑services, generating massive data. In 2020, online data reached the exabyte (EB) level; by May 2021, the database team supported more than 10 EB of storage.

Key Challenges

Diverse business types : TikTok requires graph databases for social relationships, relational databases for e‑commerce orders, and object storage for images and videos.

Rapid business growth and evolving demands : Data volume grew nearly 100× in three years, shifting requirements from tens of GB to hundreds of TB.

Massive data volume and high cost : Managing huge structured and unstructured data while controlling costs is a major challenge.

Past Architecture (2015‑2017)

The first‑generation system consisted of three layers: Application, Proxy, and Database, primarily using open‑source MySQL with sharding middleware, and relied heavily on manual operations and scripts.

Low elasticity : Scaling capacity and handling traffic spikes required extensive manual effort.

R&D efficiency issues : Long provisioning cycles and heavy operational overhead.

High overall cost : Low CPU utilization (often <10%) and poor storage efficiency (20‑30%).

Current Architecture (2021‑present)

The second‑generation platform introduced standardization, systematization, and a diversified product matrix, including MySQL, PostgreSQL, MongoDB, ElasticSearch, HTAP, and the cloud‑native distributed database veDB.

veDB follows a layered design that separates compute and storage, and further separates logs from data. It supports up to a 1:15 primary‑to‑replica ratio, enabling elastic read scaling.

Flexibility : Shared‑storage architecture allows independent scaling of compute and storage.

Compatibility : 100% MySQL 8.0, PostgreSQL 12, and MongoDB 4.0 compatibility.

High availability : Multi‑replica storage across single or three availability zones with strong consistency.

High performance : Optimized for high‑concurrency workloads, achieving QPS far beyond traditional single‑node databases.

Cost efficiency : Storage compression raises utilization from ~30% to ~70%.

Massive capacity : Single tables up to 64 TB, supporting petabyte‑scale solutions.

Business Practices

Three main instance types are supported:

Capacity‑oriented instances : For massive data volumes (e.g., e‑commerce orders) with unlimited storage scaling.

QPS‑oriented instances : Handling peak loads such as 6 million QPS during large events, with elastic compute scaling.

Small‑scale instances : GB‑level data and low QPS, optimized via virtualization and container sharing to reduce costs.

Future Outlook

Beyond 2022, ByteDance expects tens of thousands of database clusters and hundreds of thousands of instances. The focus will be on expanding the product matrix, integrating AI for intelligent operations, and further separating and integrating compute, storage, and networking resources to enhance elasticity and reduce costs.

Overall, the database team aims to simplify data management, provide unified services, and lower both development and operational costs for users.