Mastering Redis Sentinel: Automatic Failover and High Availability Explained

Redis Sentinel High Availability

What is Sentinel

Redis‑Sentinel is the official high‑availability solution recommended by Redis; it runs as an independent process that monitors multiple master‑slave clusters and can automatically promote a slave when the master fails.

Sentinel Architecture

Sentinel Functions

1) Monitoring : continuously checks the health of master and slave servers.

2) Notification : sends alerts via API when a monitored Redis instance encounters problems.

3) Automatic failover : promotes a healthy slave to master and updates other slaves to replicate the new master when the original master becomes unavailable.

Discovery and Connections

Sentinel discovers masters from a user‑provided configuration file. For each monitored master it creates two network connections: a command connection for sending commands and a subscription connection for listening to the sentinel channel.

How Sentinel discovers other slaves

Sentinel sends the INFO command to the master to obtain the addresses of all its slaves.

How Sentinel discovers other Sentinels

Sentinels announce themselves by sending a “HELLO” message (containing IP, port, and ID) over a command connection and receive others' HELLO messages via the subscription connection on the sentinel:hello channel.

Sentinels exchange health checks and can automatically discover peers monitoring the same master.

Multiple Sentinel connections

Sentinels only create command connections between each other; subscription connections are handled through the monitored master‑slave servers.

Subjective and Objective Down

Sentinel marks a server as Subjectively Down (SDOWN) when a single Sentinel deems it unreachable, and as Objectively Down (ODOWN) when a majority of Sentinels agree after exchanging SENTINEL is‑master‑down‑by‑addr messages.

Valid PING replies are +PONG, -LOADING, or -MASTERDOWN. Any other reply or no reply within the master‑down‑after‑milliseconds interval is considered invalid.

Sentinel failover steps

Detect that the master is objectively down.

Elect a leader Sentinel using a Raft‑like protocol.

If election fails, retry after twice the failover timeout.

Select a slave and promote it to master.

Send SLAVEOF NO ONE to the chosen slave.

Publish the new configuration to all other Sentinels.

Instruct the former master’s slaves to replicate the new master with SLAVEOF.

Leader Sentinel ends the failover once all slaves are replicating the new master.

Consistency guarantees

Sentinel uses a Raft‑style leader election to ensure only one leader per epoch, and configuration updates follow a last‑write‑wins rule, guaranteeing that the most recent configuration propagates to all Sentinels.

Why Sentinel can reconfigure a failed master as a new slave

Even without an active failover, Sentinel continuously rewrites the configuration of monitored instances. When a slave is promoted, Sentinel updates other slaves to follow the new master, waiting sufficiently long to receive newer configurations from peers before applying changes.

Sentinel practical guide

Environment preparation

Prepare hostnames, IP addresses, roles, and Redis ports for the master and its slaves. Example hosts: db02 as master (IP 172.16.1.52, port 6379) and several db01 instances as slaves (ports 6379, 6380, 6381).

# 1. Create Sentinel configuration directory
mkdir -p /etc/redis-sentinel/26380

# 2. Create configuration file
vim /etc/redis-sentinel/26380/sentinel.conf
port 26380
dir "/etc/redis-sentinel/26380"
sentinel monitor zls 172.16.1.52 6379 1
sentinel down-after-milliseconds zls 5000

# 3. Start Sentinel
redis-sentinel /etc/redis-sentinel/26380/sentinel.conf &

Sentinel related options

sentinel monitor mymaster 127.0.0.1 6379 2 – monitors a master named mymaster; at least two Sentinels must agree that the master is down before a failover is triggered.

sentinel down-after-milliseconds mymaster 5000 – defines the timeout after which a server is considered subjectively down if it does not reply to PING.

sentinel failover-timeout mymaster 180000 – sets the automatic failover timeout.

sentinel parallel-syncs mymaster 1 – limits the number of slaves that can sync with the new master simultaneously.

Sentinel related commands

# Check master health
127.0.0.1:26380> ping
PONG

# List monitored masters
127.0.0.1:26380> SENTINEL MASTERS

# List slaves of a master
127.0.0.1:26380> SENTINEL slaves zls

# Get master address and port
127.0.0.1:26380> SENTINEL get-master-addr-by-name zls
1) "172.16.1.51"
2) "6379"

# Manual failover
127.0.0.1:26380> SENTINEL FAILOVER zls

# Reset master configuration
127.0.0.1:26380> SENTINEL reset zls