Master Redis Sentinel: Setup, Failover, and Multi‑Sentinel Configuration

Table of Contents

What is Sentinel

Principle

Environment

Setting Up Sentinel

Slave Failure and Recovery

Master Failure and Recovery

Configuring Multiple Sentinels

1. What is Sentinel

Sentinel monitors the runtime status of a Redis system as an independent process with two main functions: monitoring the health of master and slave instances, and automatically promoting a slave to master when the master fails.

2. Principle

Single‑sentinel architecture:

Multiple‑sentinel architecture:

Multiple sentinels monitor the master‑slave pair simultaneously and also monitor each other, eliminating a single point of failure.

3. Environment

The current setup is one master with multiple slaves:

4. Setting Up Sentinel

Create a sentinel configuration file: vim sentinel.conf Insert the following line: sentinel monitor taotaoMaster 127.0.0.1 6379 1 Explanation: taotaoMaster: name of the master to monitor (customizable, may contain letters, digits, and .-_). 127.0.0.1: IP of the master. 6379: port of the master. 1: minimum number of votes required.

Start the sentinel process: redis-sentinel ./sentinel.conf Resulting console shows:

Sentinel started with ID 9059917216012421e8e89a4aa02f15b75346d2b7.

Added monitoring for the master database.

Discovered two slaves automatically (sentinel discovers slaves without explicit configuration).

5. Slave Failure and Recovery

After killing the slave process (PID 2826), Sentinel logs:

2989:X 05 Jun 20:09:33.509 # +sdown slave 127.0.0.1:6380 127.0.0.1 6380 @ taotaoMaster 127.0.0.1 6379

This indicates the slave is down. When the slave is restarted, Sentinel reports:

2989:X 05 Jun 20:13:22.716 * +reboot slave 127.0.0.1:6380 127.0.0.1 6380 @ taotaoMaster 127.0.0.1 6379

2989:X 05 Jun 20:13:22.788 # -sdown slave 127.0.0.1:6380 127.0.0.1 6380 @ taotaoMaster 127.0.0.1 6379

The slave re‑joins the replication chain; the -sdown flag indicates service recovery.

6. Master Failure and Recovery

When the master goes down, Sentinel prints a series of events:

2989:X 05 Jun 20:16:50.300 # +sdown master taotaoMaster 127.0.0.1 6379  // master down
2989:X 05 Jun 20:16:50.300 # +odown master taotaoMaster 127.0.0.1 6379 #quorum 1/1
2989:X 05 Jun 20:16:50.300 # +new-epoch 1
2989:X 05 Jun 20:16:50.300 # +try-failover master taotaoMaster 127.0.0.1 6379  // start failover
2989:X 05 Jun 20:16:50.304 # +vote-for-leader 9059917216012421e8e89a4aa02f15b75346d2b7 1  // elect leader
2989:X 05 Jun 20:16:50.304 # +elected-leader master taotaoMaster 127.0.0.1 6379
2989:X 05 Jun 20:16:50.304 # +failover-state-select-slave master taotaoMaster 127.0.0.1 6379
2989:X 05 Jun 20:16:50.357 # +selected-slave slave 127.0.0.1:6381 ... // choose slave 6381 as new master
2989:X 05 Jun 20:16:50.420 * +failover-state-wait-promotion slave 127.0.0.1:6381 ...
2989:X 05 Jun 20:16:50.515 # +promoted-slave slave 127.0.0.1:6381 ... // promote to master
2989:X 05 Jun 20:16:50.566 * +slave-reconf-sent slave 127.0.0.1:6380 ...
2989:X 05 Jun 20:16:51.333 * +slave-reconf-inprog slave 127.0.0.1:6380 ...
2989:X 05 Jun 20:16:52.382 * +slave-reconf-done slave 127.0.0.1:6380 ...
2989:X 05 Jun 20:16:52.438 # +failover-end master taotaoMaster 127.0.0.1 6379 // failover complete
2989:X 05 Jun 20:16:52.438 # +switch-master taotaoMaster 127.0.0.1 6379 127.0.0.1 6381 // master switched to 6381
2989:X 05 Jun 20:16:52.438 * +slave slave 127.0.0.1:6380 ... @ taotaoMaster 127.0.0.1 6381 // 6380 becomes slave of 6381
2989:X 05 Jun 20:16:52.438 * +slave slave 127.0.0.1:6379 ... @ taotaoMaster 127.0.0.1 6381 // 6379 becomes slave of 6381
2989:X 05 Jun 20:17:22.463 # +sdown slave 127.0.0.1:6379 ... @ taotaoMaster 127.0.0.1 6381 // 6379 down again

At this point, 6381 is the master with 6380 as its slave. Restoring 6379 yields:

2989:X 05 Jun 20:35:32.172 # -sdown slave 127.0.0.1:6379 ... @ taotaoMaster 127.0.0.1 6381 // 6379 recovered
2989:X 05 Jun 20:35:42.137 * +convert-to-slave slave 127.0.0.1:6379 ... @ taotaoMaster 127.0.0.1 6381 // 6379 becomes slave of 6381

7. Configuring Multiple Sentinels

Edit sentinel.conf again: vim sentinel.conf Add additional monitor lines:

sentinel monitor taotaoMaster1 127.0.0.1 6381 1

sentinel monitor taotaoMaster2 127.0.0.1 6381 2