Tencent Game Ops: Turning Service Delivery into Smart, Automated Microservices

Preface

The author, director of Tencent Game Operations Center, explains the goal of building a "cloud ladder" to elevate the value of game operations by focusing on massive scale, high availability, and automation, and by establishing a growth system for ops roles.

1. Redefining Ops Service

Ops service is distilled to three core functions: publishing, change, fault handling, plus SLA (security and cost). A true ops service must meet three criteria: user focus, value creation, and priceability.

User focus

Value creation

Priceability

Reasons for these criteria include moving ops from a behind‑the‑scenes role to the front, anticipating future growth, measuring impact, and enabling pricing.

2. Tencent Game Ops Service System

The service system consists of six major blocks: user experience, operation activities, cost control, consulting, version service, and overall security. These are further divided into 19 sub‑services, with three representative case studies highlighted.

3. Service Practice – Game Open/Close Server

3.1 Characteristics of Open/Close Server

Four operational phases are identified: early launch (manual monitoring), stable influx (automated opening), fine‑operation (data‑driven adjustments), and merge phase (consolidating low‑population servers).

3.2 Important Tasks for Open/Close

Tasks include manual opening, automated opening, and scheduled opening. Automation has reduced the time to open a server to under five minutes.

3.3 Advancing Open Service

Analysis of PCU versus registration curves shows the risk of "ghost servers"; the solution is to pool servers and recommend them dynamically.

Recommended servers are those that players are directed to when their current download progress is low, improving conversion.

3.4 Advancing Merge Service

Merging involves extensive data consolidation; clustering algorithms are used to balance player levels, economy, and combat power, followed by post‑merge metric tracking.

4. Extending Service to External Users – Version Service

4.1 Version Service in Daily Release

Release time has been cut from 3–4 hours (2012) to 0.88 hour (2016). The focus shifts from mere deployment speed to online recovery time, DAU impact, and revenue.

4.2 Online Recovery in Version Service

Optimizations include selecting release windows, splitting packages, pre‑download strategies, and cost‑aware distribution.

4.3 Our Gains

Online recovery time is now only 10% of its original value, and bandwidth cost has dropped by 50%.

4.4 Continuous Improvement

The service exhibits tight coupling of processes, strong reliance on ops expertise, increasing demand complexity, and ongoing cost control.

5. Introducing Microservice – Download Service

5.1 Microservice Overview

Microservices bring decentralization, decoupling, and independent evolution.

5.2 Download Service Optimization

Beyond basic functions, the service now includes intelligent throttling, user tiering, and regional controls.

5.3 Benefits

Advantages are flexibility, independent pricing, and easy extensibility.

5.4 Gift Distribution Service

To reduce download cancellations, the system segments users (WHO), determines optimal timing (WHEN) using real‑time cancellation probability, and selects appropriate gifts (WHAT) via collaborative‑filtering recommendation.

5.5 “Gift + Smart” Effect

Download completion rate increased by 8% and conversion rate by 9% without additional cost.

6. “Smart+” and Microservices

The future direction combines intelligent solutions with microservice architecture, emphasizing six capabilities—perception, analysis, decision, execution, presentation, and protection (CEBTOS)—to deliver comprehensive, secure, and cost‑effective game operations.