How QQ Album Stores 2 Trillion Photos with 300 PB and Near‑Zero Latency

QQ Album has stored more than 2 trillion photos, occupying about 300 PB of physical storage, with an average daily increase of 300 million images and 500 billion view requests, making it the world’s largest image‑based product.

Index and File Storage

The index system holds users' album lists and the list of images within each album, while the file system stores the actual image files.

Initially the service used MySQL for user indexes and a Linux file system for images, which sufficed for a small user base. As the number of users and requests grew rapidly, the team developed a custom TFS storage family with a KV engine to support the load.

Distributed KV Store (TSSD)

To meet the massive read‑write demand, TFS introduced TSSD, a distributed key‑value storage platform built on high‑capacity SSDs. It provides multi‑replica data reliability, a self‑developed persistent engine that overcomes random‑write bottlenecks, and leverages SSDs’ fast random‑read capability, achieving over 100 k IOPS per node. The system maintains 99.9% of read/write latency below 10 ms.

Index data is split into lightweight and heavyweight parts, reducing the average size of index blocks accessed per operation.

Image Compression

QQ Album supports JPG, PNG, GIF and continuously adopts higher‑ratio formats. WebP is fully supported, saving about 30% of storage and bandwidth compared with JPG. The team is also rolling out a proprietary format, sharpP, which reduces size an additional 15% over WebP.

To handle massive concurrent compression requests, a cluster of thousands of machines is deployed, largely using Docker for cost‑effective resource sharing. For latency‑sensitive scenarios, FPGA‑based compression cards are used, delivering processing times only one‑tenth of CPU‑based solutions, and this capability is offered via Tencent Cloud.

Zone Distribution

More than one billion album users are logically divided into separate zones, each equipped with a complete set of services. Zones can scale smoothly and be added on demand.

Near‑Upload Strategy

To ensure index consistency, a user’s full index is ultimately stored in a single designated zone. When a user travels to another region, uploading to the distant zone would cause high latency. The system therefore redirects uploads to the nearest zone based on geography and ISP, while synchronizing index information in the background, greatly improving upload success rates and reducing response time.

Cross‑Zone Disaster Recovery

If a zone experiences a network failure, data and indexes are written to a backup point, forming a ring‑shaped backup relationship among zones.

Download Acceleration

More than 30 CDN nodes are deployed across China, plus additional nodes in Hong Kong and the United States, providing fast download services for domestic and overseas users.

Cache Pre‑Push

Analysis shows that recently uploaded images receive the highest access frequency. After a user uploads images, the system proactively sends commands to CDN nodes to pull and cache these images, ensuring that subsequent accesses by friends load instantly without needing to fetch from the origin.

Future Work

The team will continue to improve service quality and stability, ensuring that users can safely and conveniently record their life moments with QQ Album.