Mastering Redis High Availability: Sharding, Consistent Hashing, and Sentinel Explained

1. Redis Sharding Strategy

1.1 Hash Sharding

We know that in a Redis cluster keys are distributed across shards using a hash function. For three machines a common method is hash(IP) % 3, where 3 is the number of nodes.

This simple approach is quick but has a fatal drawback: when nodes are added or removed the total number of nodes changes, causing all hash values to shift and requiring data migration.

The solution is consistent hashing.

1.2 Consistent Hashing

Consistent hashing, proposed by MIT in 1997, maps keys onto a circular space of 2^32 slots. The circle is imagined as a closed ring.

Keys (objects) are hashed onto the ring, and each node is also placed on the ring. Each key is stored on the nearest node in the clockwise direction.

If a node fails (e.g., NODE2), only the keys that were mapped to that node are moved to the next node, minimizing data movement.

When a new node (e.g., NODE4) is added, only the keys that fall into its segment are migrated, while other keys remain unchanged.

Thus consistent hashing preserves monotonicity and reduces data migration, making it suitable for distributed clusters.

When the number of nodes is small, virtual nodes (e.g., 3‑4 per physical machine) can be created to improve balance.

2. High‑Availability Solutions

Companies often provide a single Redis cluster and have two mature approaches for sharding.

Client‑Side Sharding

The client calculates the shard, whether using simple hash or consistent hashing.

This method is straightforward but requires separate implementations for each language.

Proxy‑Based Sharding

A proxy layer abstracts the sharding logic and exposes a common Redis protocol, allowing reuse across languages without the client handling high‑availability details.

The proxy routes read/write requests and embeds high‑availability logic.

3. High‑Availability Mechanisms

3.1 Redis Master‑Slave

Redis typically uses a master‑slave deployment with read‑write separation: the master handles reads and writes, slaves replicate data from the master.

During synchronization the master must generate and transfer RDB files, which can become a bottleneck if many slaves exist.

The “master‑slave‑slave” pattern mitigates this by having slaves replicate from other slaves, reducing load on the master.

3.2 Redis Sharding

Sharding distributes data across multiple instances, similar to MySQL partitioning, reducing query concentration and RDB size per node.

Each shard operates its own master‑slave replication.

Key‑to‑shard mapping can use hash(IP) % N or hash(key) % N, with algorithms like CRC16 or ASCII‑based conversion.

3.3 Redis Sentinel

What is Sentinel?

When the master fails, Sentinel nodes monitor the cluster and coordinate failover.

Detecting Master Failure

Sentinels ping masters and slaves; if a majority (N/2+1) report a master as subjectively down, it is marked objectively down.

Selecting a New Master

Sentinel evaluates slaves based on availability, priority, and replication offset, scoring them to choose the best candidate. Ties are broken by the smallest slave ID.

The elected slave announces itself as the new master to other slaves and clients.