Applying A/B Testing to Drive Growth in Tencent’s Overseas Games

Introduction: Tencent is a leader in the domestic gaming market and is expanding overseas, where differing market environments and user habits require continuous experimentation to improve user experience.

Current overseas game situation: Tencent’s overseas portfolio includes mobile, PC, and console titles, each presenting distinct challenges for A/B testing due to product type, update cycles, and connectivity requirements.

AB testing workflow: The typical process consists of user segmentation, random traffic splitting, experiment rollout, data analysis, and decision making, with typical uplift rates of 0.1‑2% but significant impact at large scale.

Challenges: Data‑localization regulations demand regional data storage, and many product teams lack a scientific experimentation mindset, leading to biased evaluations.

Multi‑cloud compliant platform: Tencent built a multi‑cloud experiment platform with regional and resource isolation, GCS bucket permissions, and IAM‑based data access controls to satisfy compliance and security requirements.

Data‑system architecture: Data is collected via SDKs, streamed through Google Cloud Pub/Sub, processed in Dataflow, stored in Cloud Storage, and aggregated with Databricks; cross‑cloud transfers are handled by Cloud Transfer Service.

Experiment system architecture: Metrics are defined, translated to a unified SQL dialect, and analyzed through offline pre‑computation and ad‑hoc queries.

Case study 1 – New‑player flow optimization: Three experiments (skip button, skip level, default view) demonstrated that removing tutorial steps harms retention, while allowing players to choose the initial view improves next‑day and three‑day retention.

Case study 2 – Email recall of churned users: Experiments on email title emphasis showed that highlighting free benefits, rather than version updates, yields higher recall rates.

Conclusion: Systematic A/B testing, supported by a compliant multi‑cloud platform and robust data pipelines, enables Tencent’s overseas games to iteratively improve user experience and drive measurable business growth.