Database Selection and TiDB Implementation in NetEase Interactive Entertainment Billing Group

The NetEase Interactive Entertainment billing group provides unified login and payment services for hundreds of channels and dozens of game studios, operating a large number of MySQL instances that have reached capacity, performance, and scalability limits.

1. Business Architecture Overview

1.1 MySQL Architecture – a keepalive‑based master‑slave setup with load‑balanced applications, VIP failover, and data backup via slave nodes, supporting dozens of deployments across 200‑400 servers and terabytes of daily data.

1.2 Current Issues – limited storage, performance bottlenecks on tables exceeding 700 GB, lack of online scalability, complex SQL due to sharding, and data silos across independent product databases.

2. Database Selection

2.1 Research Goals – compatibility with MySQL protocol, transactional support, secondary indexes, and flexible online scaling.

2.2 Candidate Solutions – evaluation of TiDB and CockroachDB (CRDB), noting TiDB’s native MySQL compatibility versus CRDB’s PostgreSQL compatibility.

2.3 Tests – comparative performance tests on 10 machines with 5 storage nodes, 160 concurrent accesses on 2 billion‑row tables. Results showed TiDB and CRDB have similar performance, with TiDB offering lower migration cost for MySQL‑centric workloads.

SELECT c FROM sbtest%u WHERE id BETWEEN ? AND ?;
SELECT SUM(k) FROM sbtest%u WHERE id BETWEEN ? AND ?;
SELECT c FROM sbtest WHERE id BETWEEN ? AND ? ORDER BY c;
SELECT DISTINCT c FROM sbtest%u WHERE id BETWEEN ? AND ? ORDER BY c;

Additional test queries included random IN, range queries, and updates on indexed and non‑indexed columns.

3. TiDB Deployment in the Billing Group

3.1 TiDB Architecture – three‑layer deployment of TiDB, TiKV, and PD modules with Nginx load balancing, supporting 41 servers, 88 instances, 20 TB of stored data, and peak QPS of tens of thousands.

3.2 Solved Requirements – eliminated capacity bottlenecks, provided HTAP capabilities, and unified data access for reporting, monitoring, and analytics.

3.3 Current Usage – production clusters run TiDB v2.1.15, handling 80 % offline big‑data tasks and 20 % online services, with latency ≤8 ms for 80 % of requests.

4. Best Practices

4.1 Cluster Management – Ansible for one‑click deployment and scaling, Docker/K8s with TiDB Operator for cloud‑native management.

4.2 Operations – Prometheus + Grafana monitoring of server resources, TiDB components, and data pipelines; handling issues such as excessive Region count and Raft store bottlenecks.

4.3 Full‑Network Database Traversal – TiDB enables querying across all product data from a single source, removing the need for heterogeneous source maintenance.

4.4 Data Migration – full‑load via Mydumper/Loader or Lightning, incremental sync via Syncer or DM, and reverse sync using TiDB Binlog for backup and cross‑system replication.

4.5 Simplified Sharding – TiDB allows querying massive partitioned tables as a single logical table, reducing join complexity and improving query latency.

4.6 Business Migration – stepwise approach: data sync, read/write verification, gray‑scale traffic shift, and final cut‑over, ensuring data integrity, minimal downtime, and rollback capability.

5. Summary and Outlook

TiDB’s MySQL compatibility, strong transactional guarantees, and horizontal scalability effectively address the billing group’s high‑capacity and high‑availability needs, while ongoing improvements such as better backup mechanisms, larger transaction support, and hotspot mitigation are desired.