Replication‑Free Failover for RocketMQ: Achieving Second‑Level Takeover Without Data Copy

Alibaba Cloud's messaging team reports that their paper "Replication‑Free Failover: Protocol‑Fenced Takeover for Stateful Services" was accepted to the ACM FSE 2026 Industry Papers Track. The work targets large‑scale, multi‑tenant RocketMQ clusters and proposes a failover solution that balances low cost, high steady‑state performance, and rapid recovery.

Key Innovations

No extra business‑data replication, normal data path unchanged. The design keeps a single‑replica, single‑node file system with cloud block storage that already provides storage‑side redundancy, avoiding additional write amplification and preserving compatibility with upstream RocketMQ.

Protocol‑level safe takeover with verifiable ownership transfer. Multi‑Attach makes the disk visible to candidate nodes early; NVMe Persistent Reservation (NVMe PR) moves write‑permission control to the storage protocol layer. The solution adds holder verification, atomic preemption, critical‑stage marking, and back‑off mechanisms to prevent write‑permission races during filesystem recovery.

Consistency loop for local file‑system takeover. The process confirms write permission, clears old caches on the takeover node, mounts the filesystem, replays filesystem logs, recovers the crashed application, and finally restores traffic through a readiness check, ensuring safe state restoration when a shared disk becomes visible to a new node.

Decentralized fault detection based on persistent progress. By examining lease version progress stored on the shared disk, the system determines whether the current primary can still persist data, detecting I/O hangs where the node is alive but persistence has stopped. Storage‑protocol write isolation guarantees single‑writer safety during network partitions.

Minimal‑privilege, auditable production‑grade implementation. High‑privilege operations such as NVMe PR, cache handling, and mounting are confined to a node‑side agent, so business containers run without system‑level rights. Fault detection, write‑permission switching, application recovery, and traffic restoration are chained into an observable, auditable, repeatable end‑to‑end recovery flow.

Unlike approaches that redesign the storage engine, introduce shared WALs, or move to object storage to achieve stateless nodes, this solution retains RocketMQ's existing single‑node filesystem and read/write semantics, only redesigning the failover path. Consequently, it does not replace the normal storage engine, allowing reuse of mature CommitLog, filesystem cache, and crash‑recovery mechanisms, and avoids extra refactoring for future storage features.

Evaluation on Apache RocketMQ shows that node‑failure recovery time shrinks from the typical minute‑level latency of Kubernetes single‑replica cloud‑disk setups to seconds, while preserving throughput and latency close to native single‑replica deployments. Production validation includes tens of thousands of disaster‑recovery drills and continuous operation in a live environment, demonstrating the approach's suitability for single‑primary, single‑writer stateful services.

With growing AI‑infra demands for reliable messaging and stateful services, the replication‑free failover technique offers a new engineering choice for cloud‑native infrastructure, delivering higher stability, efficiency, and lower cost.