In-depth Analysis and Practice of Tencent Cloud EB-level Object Storage Architecture

Background: On May 23‑24, the Tencent "Cloud + Future" summit was held in Guangzhou under the theme "Huanqi". Government leaders, academic experts, industry veterans and technical experts discussed cloud computing and digital industry innovation.

At the summit, Liu Jinming, Deputy Director of Tencent Cloud Storage Business Center, delivered a talk titled "Deep Analysis and Practice of Tencent Cloud EB‑level Object Storage Architecture". The following is a summary of his presentation.

Tencent’s self‑developed storage platform TFS was launched in 2016, providing storage for internal products such as Photo, QQ, WeChat, Weiyun and Tencent Video. The storage capability was opened to external users in 2013. By 2014 the platform exceeded 500 PB, reaching trillion‑level data volume. In 2015 the engine was upgraded to a commercial standard, achieving EB‑level data volume. After further engine and architecture optimizations in 2016, performance improved and intelligent data analysis on top of storage emerged.

COS (Cloud Object Storage) product ecosystem is divided into three layers:

COS Storage Layer : Provides SDKs for multiple languages and an S3‑compatible API. Application Access Layer : Offers advanced capabilities such as image moderation, video transcoding, intelligent security, big‑data and medical imaging solutions. CDN Acceleration Layer : Enables high‑traffic customers (e.g., video, mobile app stores) to accelerate upload and download via CDN.

COS is now available in 13 global regions across Asia, Europe and the Americas, with a Russia region launching soon.

The architecture follows a typical three‑tier model:

Network Access Layer – multi‑level BGP access, intelligent load balancing via TGW, and DDoS cleaning. Application Access Layer – stateless, horizontally unlimited scaling. Data Layer – cross‑rack and cross‑switch deployment, modular index and data expansion, hot‑cold data intelligent scheduling.

Key service requirements highlighted were: high stability, data loss prevention, strong security, excellent performance, and low cost – essentially “high quality at low price”.

Reliability is ensured through automatic fault isolation, rate limiting, flow control, and a 99.95 % availability SLA. For predictable spikes (e.g., holiday video uploads, e‑commerce promotions), capacity can be pre‑scaled, hot data distributed, and non‑essential features temporarily disabled or degraded (e.g., lower video bitrate).

Data durability reaches 11 nines via multi‑replica and erasure coding, with rapid fault detection and reconstruction across disks and data centers.

Security is addressed with end‑to‑end encryption covering transmission, access and storage.

Use cases include large‑scale media production: a collaboration with Tencent Pictures required transferring hundreds of terabytes of raw footage worldwide. COS enabled fast, secure storage and cross‑region replication, eliminating the need for physical hard‑drive shipments.

Performance optimizations include:

Chunked concurrent upload for large files.

Protocol and engine tuning for small files to reduce latency.

CDN‑proxied upload acceleration.

Image formats such as Guetzli or WebP to reduce size without quality loss.

Cost control features:

Three storage classes (Standard, Infrequent Access, Archive) with lifecycle policies for automatic tiering.

Potential cost reduction up to 80 % with optimal tier selection.

Additional advanced features:

Lifecycle management – rule‑based data migration and deletion.

Cross‑region replication – improves reliability and speeds up global data distribution.

Versioning – protects against accidental deletions.

Tagging – enables data classification for downstream processing (e.g., live‑stream categorization, bitrate labeling).

Inventory (list) – generates periodic object listings for big‑data analysis or billing.

Solution examples:

Smart security – video streams from cameras uploaded to COS, followed by cloud‑side decoding, face detection/comparison, and real‑time alerts.

Big‑data integration – HDFS‑to‑COS migration tool and Tencent Cloud big‑data suite allow real‑time and batch analytics on stored data.

Future plans include more application‑oriented solutions for customers.

Overall, Tencent Cloud COS combines a robust three‑tier architecture, high reliability, strong security, performance optimizations, and flexible cost models to meet diverse enterprise storage needs.