TiDB Adoption and Operational Practices at Dewu
Dewu transitioned from a single‑instance MySQL setup to a distributed TiDB architecture—using three TiDB, PD, and TiKV/TiFlash nodes, automated backups, Prometheus monitoring, and chaos‑engineering controls—which delivered linear scalability, eliminated slow queries, cut infrastructure costs by about 58 % and maintained stable performance during major sales events.
Dewu originally relied on the open‑source relational database MySQL. As business grew, single‑instance, single‑database, and single‑table designs hit performance and storage limits, prompting a gradual migration toward the distributed relational database TiDB.
Since early 2020, TiDB has been introduced across multiple business lines. The architecture uses an SLB for load‑balancing, with TiDB Server and PD deployed on SSD‑less machines and TiKV on local‑SSD machines, balancing performance and cost.
MySQL vs. TiDB comparison
Scalability : MySQL relies on vertical scaling and sharding, which requires significant development effort. TiDB’s compute‑storage separation and distributed TiKV enable straightforward horizontal scaling, though TiKV rebalancing must be monitored.
Performance : MySQL offers lower latency for point queries due to its single‑node nature, while TiDB excels at high‑throughput and aggregation queries thanks to its distributed coprocessor.
Resource cost : For small, simple workloads MySQL is cheaper; for large, complex workloads TiDB can reduce overall cost by up to 58% compared to a MySQL sharding cluster.
Operational cost : MySQL benefits from mature ecosystem and simple master‑slave failover. TiDB’s distributed design lacks a master‑slave model, making failure isolation more complex and requiring comprehensive monitoring.
DDL limitations : TiDB supports online schema changes but has restrictions such as single‑operation ALTER statements, limited index types, and constraints on column type modifications.
Operational practices
Selection criteria focus on scenarios where TiDB’s strengths (high throughput, complex queries, large data volume) outweigh MySQL’s low‑latency advantage. Deployment specifications include 3 TiDB nodes, 3 PD nodes, and 3 TiKV/TiFlash nodes with baseline and high‑performance configurations.
Backup uses the official BR tool with daily full backups retained for 7 days. Monitoring integrates Prometheus alerts into an internal platform (OneDBA) for alert silencing, deduplication, and closed‑loop handling.
Change management enforces DDL/DML type checks, syntax validation, compliance rules, and risk assessment before execution. Stability governance includes automated chaos‑engineering fault injection (8 scenarios) using ChaosBlade, and regular health inspections.
Core business scenario
The order database faced performance bottlenecks due to sharding by merchant ID, leading to slow queries and high load. After evaluating options, Dewu adopted TiDB as a distributed solution.
Compatibility and performance testing
All production SQL statements were replayed on TiDB, achieving 100% compatibility. Performance benchmarks showed TiDB’s response time (RT) was higher than MySQL for low‑concurrency workloads but narrowed with increased concurrency, and TiDB scaled linearly with added nodes.
Example SQL statements used in testing:
SELECT * FROM table_name WHERE xx_id=1203030 AND xx_status IN(4000,3040) AND is_del=0 ORDER BY id DESC, create_time DESC LIMIT 20; SELECT [column_name] FROM table_name WHERE xx_id=1203030 AND xx_status=8010 AND close_type=13 AND close_time > '2022-06-19 18:16:24.519' LIMIT 0,30000; SELECT * FROM table_name WHERE xx_id=1203030 AND xx_status IN(7000,8000,8010) AND is_del=0 ORDER BY id DESC, create_time DESC LIMIT 20; SELECT count(*) FROM table_name WHERE seller_id=1203030 AND is_del=0 AND biz_type IN('0','12') AND create_time >= '2021-11-01 00:00:00' AND create_time < '2021-11-30 23:59:59' AND (xx_status <> 1000 OR biz_type <> 21);Issues encountered
Optimizer mis‑plan due to complex conditions and index overload; resolved via SQL review and SPM binding. Bugs in TiDB 5.3.2 caused write‑write conflicts and region unavailability; mitigated by upgrading to newer versions and careful leader handling.
Results
After gradual traffic migration, slow queries virtually disappeared. Cost decreased by 58% (from 10 MySQL write + 10 read instances to 6 TiDB + 12 TiKV instances). The system handled major sales events (Double 11, Double 12) with stable response times.
Conclusion
The article summarizes Dewu’s practical experience with TiDB, offering a reference for enterprises considering distributed databases. The methodology can be generalized to other large‑scale data‑introduction projects.
DeWu Technology
A platform for sharing and discussing tech knowledge, guiding you toward the cloud of technology.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.