Mastering Redis Sentinel: Setup, Communication, and Automatic Failover

Sentinel Architecture

Before Redis 3.0, clustering relied on Sentinel to monitor the master node and perform automatic failover, but its configuration was complex and performance limited, with brief access interruptions during master‑slave switch‑over and a single master that could not handle high concurrency.

Sentinel Initialization

Sentinel is a high‑availability solution composed of one or more Sentinel instances that monitor one or more Redis masters and their slaves. When a monitored master goes offline, a slave is promoted to master and the former master becomes a slave of the new master.

Sentinel itself runs as a special Redis server and can be started with: redis-server xxx.conf --sentinel Example experimental environment (all services on localhost):

127.0.0.1:6379 – Master

127.0.0.1:6380 – Slave

127.0.0.1:6381 – Slave

127.0.0.1:26379 – Sentinel

127.0.0.1:26380 – Sentinel

127.0.0.1:26381 – Sentinel

# sentinel port
port 26381
# run in background
daemonize yes
# monitor master with one slave
sentinel monitor mymaster 127.0.0.1 6379 2

Sentinel start commands:

redis-server redis-sentinel-26379.conf --sentinel
redis-server redis-sentinel-26380.conf --sentinel
redis-server redis-sentinel-26381.conf --sentinel

Query cluster status on a Sentinel node with the info command.

Communication Between Sentinel and Redis

Command connection : Sentinel periodically sends info (default every 10 s, 1 s during failover) and a PING each second to check if the server is online.

Subscription connection : Sentinel subscribes to the __sentinel__:hello channel of the master to receive heartbeat messages.

Sentinel‑to‑Sentinel Communication

All Sentinels monitoring the same master broadcast a message to the __sentinel__:hello channel every two seconds to announce their presence.

Sentinels do not create subscription links between each other; they discover each other through the monitored Redis servers.

How Sentinel Discovers Redis Nodes

Sentinel sends info to the master every 10 seconds and parses the reply to obtain server information (run_id, role) and the list of slaves, e.g.

slave0:ip=127.0.0.1,port=6381,state=online,offset=2712947,lag=0

. This allows automatic discovery without manual configuration.

Sample INFO output from a slave shows fields such as run_id, role, master_host, master_port, master_link_status, slave_priority, and slave_repl_offset.

Service Down Detection

Sentinel distinguishes subjective down (a single Sentinel marks a master as down) from objective down (a majority of Sentinels agree). The down-after-milliseconds setting defines how long a master must fail to respond before being considered subjectively down.

Leader Sentinel Election

All online Sentinels are eligible.

Each election increments the configuration epoch.

Within an epoch, a Sentinel can be set as a temporary leader; once set, it cannot be changed in that epoch.

A Sentinel that detects an objective down requests other Sentinels to set it as temporary leader.

The first Sentinel to receive a majority of such requests becomes the leader.

If no leader is elected within the timeout, a new election starts.

Failover Process

Failover consists of three steps:

Select a slave to promote to master.

Make all remaining slaves replicate the new master.

Reconfigure the old master as a slave of the new master.

Selecting the New Master

The selection algorithm filters out offline slaves, removes slaves that have not responded to Sentinel’s info for more than five seconds, discards slaves whose link to the master has been down longer than down-after-milliseconds × 10, then sorts the remaining candidates by slave_priority. If priorities tie, the slave with the highest slave_repl_offset (most up‑to‑date data) is chosen; if still tied, the smallest run_id wins.

Updating Slave Replication Targets

After a new master is elected, the leader Sentinel instructs other slaves to replicate the new master using the saveof command.

Demoting the Old Master

The final step sets the previously down master as a slave of the new master, completing the failover.