How Alibaba’s Next‑Gen Database Powered Double 11: Elasticity, Cloud & AI

Introduction

Alibaba researcher Zhang Rui, head of the Alibaba Group Database Technology team, discusses how the new generation of database technologies supported the massive traffic of the Double 11 shopping festival.

Extreme Elasticity for Double 11

To handle peak loads, the system must provide high performance, low cost, and strong stability. Elasticity is achieved by moving databases to the cloud, using high‑performance ECS instances with SPDK/DPDK and NVMe storage, and deploying a hybrid‑cloud management system that quickly builds mixed‑cloud environments before the event. VPC networking connects internal and public clouds to solve inter‑cloud connectivity.

Elastic Scheduling and Storage‑Compute Separation

Beyond cloud migration, Alibaba uses offline‑online mixed deployment, containerization, and storage‑compute separation to further reduce costs and improve elasticity. High‑speed 25 Gbps RDMA networks and a distributed storage system (Pangu) enable sub‑millisecond latency and high availability.

Response latency: single‑path read/write 0.4 ms, RDMA <0.2 ms.

Asynchronous third replica improves latency stability.

QoS flow control balances IO based on front‑end load.

Fast failover within 5 seconds.

Four‑rack deployment achieves nine‑9s reliability.

Additional optimizations include redo‑log sync (100 % throughput increase) and disabling double‑write buffer (20 % throughput boost, 100 % bandwidth saving).

X‑DB Architecture

The next‑generation X‑DB incorporates the Paxos distributed consensus protocol for strong consistency across regions, supporting three‑node deployments that match single‑node performance while tolerating network jitter.

Core Technology 1: X‑Paxos

High‑performance Paxos library enabling cross‑AZ/Region strong consistency with >99.999 % availability.

Core Technology 2: Batching & Pipelining

Batching and pipelining of transaction logs allow out‑of‑order receipt while preserving order on commit, maintaining high throughput under high latency.

Core Technology 3: Asynchronous Commit

Database thread pools use asynchronous commit to keep threads busy, further boosting throughput in three‑node mode.

Performance Benchmark

In a three‑node same‑IDC deployment, X‑DB achieved 2.4× the insert throughput of MySQL Group Replication with lower latency. In cross‑region tests, X‑DB’s insert throughput was 5.94× higher (50.4 k vs 0.85 k) and latency 38 % of the baseline.

X‑KV Enhancements

X‑KV, built on MySQL’s Memcached plugin, now supports more data types, non‑unique and composite indexes, multi‑get, and online schema changes via TDDL, delivering ultra‑fast reads, strong consistency, and lower migration cost.

Transaction Seller DB Bottleneck Solutions

To reduce synchronization delay and improve complex query performance, Alibaba introduced independent queues for large sellers, merged sync links, and rate‑limiting, alongside a hybrid deployment strategy.

ESDB – Distributed Document Store

ESDB extends Elasticsearch with a SQL interface, enabling seamless migration from MySQL and providing dynamic secondary hashing for large sellers, solving data‑sync bottlenecks while offering rich query capabilities.

Database Monitoring Evolution

The monitoring stack progressed from Agent + MySQL to Agent + DataHub + distributed NoSQL, and finally to Agent + real‑time compute engine + HiTSDB, handling up to 14 million metrics per second with low‑latency aggregation.

CloudDBA – AI‑Driven Database Management

CloudDBA combines DBA expertise, big‑data analytics, and machine intelligence to automate diagnosis, optimization, and operation, aiming for a future where machines manage databases autonomously.

Outlook for Next Double 11

Three guiding principles: Higher – support higher transaction peaks with lower cost; Faster – continuously improve system speed; Smarter – apply machine intelligence across databases, scheduling, recommendation, and customer service.