Low‑Latency and High‑Availability Design of RocketMQ for Double‑11 Peak Traffic

By briefly reviewing the development history of Alibaba's middleware (Aliware) message engine, the article starts with the low‑latency challenges encountered during the Double‑11 shopping festival, illustrating problems such as slow response, avalanche effect, poor user experience, and transaction decline, and explains why a low‑latency, high‑availability solution is essential.

The evolution of the message engine is divided into three generations: the first generation used a push model with relational databases, the second generation adopted a pull model with a proprietary storage comparable to Kafka but prioritized reliability, and the third generation introduced RocketMQ in 2011, a hybrid push‑pull engine that has been open‑sourced, proven in six years of Double‑11 core‑transaction tests, and now serves thousands of Alibaba applications with trillion‑level message traffic.

Section 2 explores low‑latency and availability. It defines throughput and latency, explains how high latency can cause request buildup (Little’s law) and lead to system avalanche. It then details the specific latency sources in RocketMQ: JVM pauses (GC, JIT, biased‑lock revocation), lock contention, memory management (direct reclaim, page cache pressure), and page‑cache‑related I/O delays.

For each source, the article presents concrete mitigation techniques: GC tuning (heap size, GC flags, logging to tmpfs, disabling shared memory stats), using CAS to eliminate locks, adjusting Linux kernel parameters (vm.extra_free_kbytes, vm.swappiness), pre‑allocating memory, warming files, mlock, read‑write separation, and other optimizations that together eliminated high‑latency write spikes during the latest Double‑11.

Section 3 introduces the three "capacity‑guarantee" mechanisms: rate‑limiting (leaky‑bucket, token‑bucket), degradation, and circuit‑breaker. It explains how these techniques protect core services from traffic bursts, prevent avalanche, and maintain SLA, citing examples such as Guava RateLimiter, Netty TrafficShaping, and Netflix Hystrix.

Section 4 describes the high‑availability solution based on multi‑replica deployment across data centers. It outlines the CAP trade‑offs, compares common HA patterns (cold standby, Master/Slave, Master/Master, two‑phase commit, Paxos), and focuses on the Master/Slave model used by RocketMQ, detailing synchronous vs. asynchronous replication, consistency, latency, and fault‑recovery characteristics.

Section 5 details RocketMQ's HA architecture: Zookeeper stores persistent and ephemeral nodes for master‑slave state, a stateless HA Controller observes state changes and drives the finite‑state machine (single‑master → async replication → semi‑sync → sync replication). The controller ensures automatic failover within seconds without operator intervention.

Section 5.1 explains availability metrics (MTBF, MTTR) and the “N‑nines” concept, while Section 5.2 shows how RocketMQ’s HA design shortens MTTR and improves overall availability.

The outlook notes ongoing work to further reduce storage latency, support cross‑language calls, and build a fourth‑generation engine with multi‑level QoS for emerging IoT, big‑data, and VR scenarios, continuing the open‑source contribution philosophy.