A Decade of Alibaba Database Evolution for Double 11: From Commercial to Open‑Source to Self‑Developed

Alibaba's database technology has progressed through three distinct stages: 2005‑2009 commercial databases (IOE era), 2010‑2015 open‑source databases (AliSQL, TDDL middleware) and 2016‑present self‑developed databases (X‑DB), each driven by the growing demands of the Double 11 shopping festival.

The "going‑away‑from‑IOE" effort replaced Oracle, IBM mainframes and EMC storage with distributed middleware, AliSQL and high‑performance X86/SSD servers, achieving horizontal scalability, autonomous feature development, and improved stability via multi‑active, cross‑city architectures.

From 2016 the team built X‑DB to provide a globally distributed, Spanner‑like database with high read/write throughput, cost‑effective SSD storage, and emerging hardware such as NVM, FPGA and RDMA, while also adopting cloud ECS resources and hybrid‑cloud VPC networking for elastic capacity.

Key innovations included online‑offline mixed deployment (storage‑compute separation), intelligent self‑driving database features (SQL auto‑optimization), full‑link pressure testing with shadow tables, and a second‑generation monitoring system using the TSDB time‑series database and real‑time full‑SQL capture.

Cache integration via Tair and the KV component X‑KV, along with dynamic hotspot redistribution, further reduced latency; storage‑compute separation was realized using Ceph and the proprietary Pangu 2.0 storage, complemented by 25 Gbps TCP and RDMA networks and the DBFS zero‑copy I/O path.

Pre‑heating of data and applications, JIT warm‑up, and redundant dual‑link pipelines ensured a smooth transaction curve at the midnight of Double 11, while the CloudDBA service extended these optimizations to external customers.

The article concludes with a personal reflection from the author, Zhang Rui, on the relentless technical challenges and achievements that have enabled Alibaba to continuously push the limits of database performance for the past decade of Double 11 events.