How Consistent Hashing Minimizes Data Migration During Auto-Scaling

- Background -

Microservice architectures are booming, and Docker has become mature; together they form a perfect match. In any business scenario, containerized services face dynamic scaling, and automated addition and reclamation of container resources can ease operational pressure.

Before and after scaling, load balancing is required to keep node load balanced, making scaling more "elegant". Load‑balancing algorithms often involve the consistent hashing algorithm.

- Remainder Hash Algorithm -

Each node in a service cluster has a "hash address" as a unique identifier, calculated as: add = hash(object) mod N When scaling adds or removes a node, the address mapping of the remaining nodes changes: add = hash(object) mod (N±1) This causes data migration across all nodes, which is costly.

- Consistent Hashing Algorithm -

Consistent hashing arranges the hash value space into a ring ranging from 0 to 2^32‑1, using the node's IP or hostname as the identifier.

Assume we have three nodes and four data items distributed on the ring:

According to consistent hashing, each data item binds to the nearest node in the clockwise direction: data A to node 01, data D to node 02, and data B and C to node 03.

If node 03 is removed, data B and C re‑bind to node 01, while other data remain unchanged.

When a new node 04 is added, only data B needs to migrate to node 04.

- Summary -

Consistent hashing, with its special data structure and binding algorithm, dramatically reduces the amount of data migration when nodes are added or removed, shortening the time required for dynamic scaling.

Most market solutions achieve scaling in minutes; only a few claim "second‑level" scaling. Faster scaling reduces the risk of service outages or cascading failures during large‑scale business events.