How Tencent Built YottaStore: A Cloud‑Native Storage Revolution

Prologue

Behind Tencent's growth, users see heartfelt products, while unseen are the foundational services that constantly safeguard operations.

Born for the Cloud

From the QQ album era, Tencent's storage service grew with the PC internet wave and evolved alongside Tencent's consumer products, repeatedly being refined as data volumes surged.

Since 2017, the internet entered its second half, with industrial internet waves reshaping storage scenarios for both consumer and enterprise use. Cloud demand exploded, putting unprecedented pressure on Tencent Cloud's infrastructure, especially cloud storage.

Breaking the Mold

At the end of 2017, a massive influx of users revisited old QQ photos, causing a sudden surge of cold‑data access on the TFS (Tencent File System) platform and a subsequent upload spike when those photos were shared on Moments.

During this period, the TFS storage engine operated in a small SET mode (12 servers per SET). Sudden traffic spikes forced operations to constantly toggle SET switches to keep request volumes within limits, but the underlying data could not be rebalanced across SETs, leading to escalating load.

In the cloud era, such issues multiplied. TFS, designed in 2006 for the then‑available data center scale, could no longer meet the unpredictable, large‑scale demands of B‑to‑B cloud scenarios.

By 2015, COS (Object Storage) was built on TFS, but its rapid growth soon outpaced expectations, requiring ever‑increasing budgets and facing fierce competition.

Resolute Reform

The team decided to split efforts: one group focused on stability, the other on building a new system. This dual‑track approach eventually proved effective.

Key outcomes:

The stability team kept the existing system running, reduced costs, and ensured TFS supported the critical YottaStore handover period.

The YottaStore team, though small and newly formed, worked intensely and delivered the new system on schedule.

Glorious Years

After more than a year of development, YottaStore launched in 2019. The first public‑cloud AZ (Availability Zone) cluster in Guangzhou marked the first cross‑AZ storage capability for Tencent Cloud, achieving 100% availability despite network and hardware issues.

YottaStore's capabilities include:

Cluster scale: a single cloud‑native cluster can manage tens of millions of servers, with metadata requiring only ~600 GB of storage on a single machine.

Resource utilization: real‑time reclamation pushes disk usage above 90%, supporting multiple redundancy modes and heterogeneous hardware (JBOD, HDD, SSD, tape) without additional development effort.

Operational excellence: high availability, cost efficiency, and seamless support for major Tencent services such as WeChat, QQ, Weishi, and Tencent Video, as well as tens of thousands of external customers via COS.

Impact and Recognition

YottaStore’s success solidified Tencent Cloud’s position in the cloud storage market, delivering lower storage costs while maintaining high service quality, attracting new customers, and earning the 2021 Tencent Annual Technology Breakthrough Award.

Conclusion

In internal YottaStore retrospectives, the team emphasizes two "firsts": a first‑heart and a first‑sight, committing to continue building the world’s largest and best storage system.

Images illustrating the story: