Tencent Game Big Data Operations Overview and Asset Management Practices

Author Chen Cai from Tencent shares an overview of the company's game big‑data operations, emphasizing daily transmission of 17,000 billion records (~260 TB) and a total storage exceeding 100 PB, accounting for 20 % of the company's data.

The operation faces numerous pain points: heterogeneous data lacking unified standards, inconsistent calculation logic, complex full‑link quality issues, low data quality, fragmented data without cost management, difficulty evaluating data value, and security/compliance concerns.

After six years of iterative solutions, Tencent built a game‑data asset‑management system that integrates organization, processes, policies, and technology to guarantee data delivery quality, efficiency, cost, and safety.

Metadata Management includes heterogeneous adaptation with centralized storage, open business metrics, descriptive data for business understanding, automatic lineage construction, and extensibility to support operations.

Data Collection Architecture supports single‑machine protocols (UDP 5 W qps, KCP 9 W, TCP 10 W), 180 Mbps per node throughput, distributed high‑availability with decoupled scaling, WAL double‑backup, 99.99 %+ completeness, multi‑protocol and multi‑format support, professional data management, and comprehensive peripheral system integration.

Data Standard Management is illustrated with visual diagrams (images omitted for brevity).

Data Governance Challenges are threefold: complex end‑to‑end data service links make issue localisation hard, business service failures hinder data back‑trace, and platform anomalies require rapid impact assessment.

Solution : Build a full‑link quality assurance system based on data‑and‑business lineage.

Lineage Construction creates data‑business relationships to achieve SLA of 99.99 %+ and enables alerts and impact evaluation within one minute.

Asset Operation – Cost Management and Lifecycle Management address uncontrolled data growth, rising costs, low compute efficiency, and maintenance difficulty; a mathematical model y(t)=f(V(t), I) describes data online degree decay over time.

Asset Operation – Data Security Management outlines four key measures for game data security (images omitted).

Data‑Driven Value Assessment explains the importance of ROI calculation, cost optimization, and deep‑mining data for refined scenarios, highlighting challenges such as lack of industry standards, internal quantification methods, and scenario‑specific value differences.

The article concludes with a summarized practice workflow (visual diagram omitted) and encourages readers to like, bookmark, and share the content.