Optimizing Redis Cluster Slot Migration to Reduce Latency and Improve High Availability

Redis clusters are widely deployed for scalability and high availability, but during horizontal scaling the native slot‑migration mechanism often causes severe latency spikes, service interruptions, and even node failures.

Problem analysis reveals four main issues: (1) the per‑key migration process blocks the single worker thread with costly serialization, network transfer, and acknowledgment steps; (2) the ask‑move redirection doubles client round‑trips and breaks pipelines or Lua scripts; (3) each migrated slot triggers a full topology update, overwhelming the cluster with computation and network traffic; (4) migration state is not replicated to slaves, so failover during migration breaks the process and can cause data inconsistency.

Serialize the key‑value pair

Send the serialized packet over the network

Wait for the target to acknowledge receipt and load

Delete the local replica and free memory

To address these problems the authors redesign the migration using Redis’s existing master‑slave replication mechanism. Instead of moving each key individually, the target node is presented as a slave of the source, allowing bulk slot data to be transferred via the RDB stream while preserving consistency.

Implementation details include adding slot‑information exchange to the sync protocol, restructuring the RDB file so that slot data are stored sequentially with offset metadata at the file tail, and enabling the target node to incrementally load each received network packet rather than waiting for the whole RDB file. This keeps the single‑threaded architecture intact while off‑loading heavy work to background I/O.

Effect analysis shows that the new approach dramatically reduces latency impact (the source continues to serve requests, and the target loads data piece‑wise), eliminates ask‑move redirection, batches topology changes into a single update after multiple slots are migrated, and propagates migration state to slaves, thereby providing high‑availability during failover.

Performance testing on three identical physical machines compared native migration with the custom solution using 100 KB and 1 MB hash data sets. The custom migration kept request latency stable throughout the transfer, while the native method exhibited sharp latency spikes and occasional node time‑outs.

Conclusion and outlook – The redesigned slot migration significantly improves Redis cluster stability and operational efficiency, though it introduces higher bgsave pressure and increased memory consumption during transfer. Future work will focus on mitigating these side effects and further optimizing the migration pipeline.