Mastering Redis Cluster: Hash Slots, Gossip Protocol, and Jedis for High Availability

1. Interview Focus and Background

Redis has evolved to support native clustering, allowing multiple instances across machines to store partitions of data while providing automatic failover from a primary to its replica.

2. Data Distribution Algorithms

Redis Cluster uses a three‑step approach: basic hash, consistent hashing (as used in Memcached), and the Redis‑specific hash‑slot algorithm. The hash‑slot mechanism assigns each key to one of 16,384 slots by computing a CRC16 checksum and taking the modulo 16384. Slots are evenly distributed among master nodes, so adding or removing a master simply moves a subset of slots.

Clients can force keys into the same slot using hash tags, e.g., SET mykey1:{100} value and SET mykey2:{100} value, ensuring both keys reside on the same node.

3. Redis Cluster Architecture

Each master node stores a portion of the keyspace and opens two ports: the standard client port (e.g., 6379) and a cluster‑bus port (client port + 10000, e.g., 16379). The cluster‑bus port handles internal node‑to‑node communication for configuration updates, failure detection, and failover authorization.

4. Internal Communication Mechanism

Nodes exchange messages using a binary gossip protocol, which reduces bandwidth and processing overhead. Every second each node sends ten ping messages to five peers that have not communicated recently. Pings carry the node’s status and a subset of other nodes’ metadata. If a node fails to respond within cluster_node_timeout, it is considered subjectively failed (pfail). When a majority of nodes agree, the failure becomes objective (fail).

Message types include:

meet : introduces a new node to the cluster.

ping : health check and metadata exchange.

pong : response containing status and metadata.

fail : notifies peers that a node is down.

5. Gossip Protocol Details

The gossip protocol disseminates metadata such as node failures, additions, removals, and hash‑slot assignments. Because updates are spread across nodes, the load on any single node is reduced, though there is a slight delay before the entire cluster converges.

6. Jedis Client Implementation for Redis Cluster

Jedis, the popular Java Redis client, implements a smart cluster client:

Redirect‑based client : When a command is sent to any node, the node computes the key’s hash slot. If the slot belongs to a different node, the server returns a MOVED response, prompting Jedis to redirect the request.

Automatic redirection : Using redis-cli -c or Jedis’ built‑in logic, the client follows MOVED replies automatically.

Local slot cache : Jedis maintains a hashslot → node map, reducing the need for repeated redirections.

Handling ASK redirects : During slot migration, a node may return ASK. Jedis follows the temporary redirect without updating its slot cache until the migration completes and a MOVED is received.

Retry logic : Jedis retries up to five times before throwing JedisClusterMaxRedirectionException.

Recent Jedis versions have optimized excessive slot‑cache updates and ping traffic, mitigating the high‑IO overhead seen in older releases.

7. High Availability and Failover

Redis Cluster’s failover mirrors Sentinel’s logic:

When a master is deemed pfail, other nodes broadcast the status via gossip.

If a majority confirms, the master is marked fail.

One of its replicas is promoted to master, provided it has not exceeded cluster-node-timeout * cluster-slave-validity-factor without communication.

Replica election considers replication offset (more up‑to‑date replicas are preferred) and slave priority.

Overall, Redis Cluster integrates replication and Sentinel‑style failover into a single, native solution.

Reference

Content derived from technical interview preparation material on Redis Cluster architecture and Jedis client behavior.