How to Build a Redis Master‑Slave + Sentinel Floating VIP HA Solution on AWS

1. Background

The company heavily uses Redis; high‑traffic services run on a Codis cluster and a Redis 3.0 cluster deployed on AWS EC2. The cluster handles over 500k QPS and is already highly available and horizontally scalable, but some smaller services only need a single instance with high availability.

Various solutions exist (Redis master‑slave + LVS VIP, master‑slave + Sentinel VIP, custom failover logic). This article introduces the Redis master‑slave + Sentinel floating VIP approach, which will be used at scale.

2. Environment

#redis
100.10.32.54:6400 master
100.10.32.55:6400 slave
100.10.32.250 VIP
#sentinel
100.10.32.54:26400 sentinel local node
100.10.32.55:26400 sentinel local node
100.10.32.57:26400 sentinel arbitration node

3. Deployment

1) Install Redis yum -y install redis 2) Create Redis configuration files for the master (100.10.32.54) and slave (100.10.32.55)

vim /etc/redis_6400.conf

daemonize yes
pidfile "/var/run/redis_6400.pid"
port 6400
tcp-backlog 65535
bind 0.0.0.0
timeout 0
tcp-keepalive 0
loglevel notice
logfile "/var/log/redis/redis_6400.log"
maxmemory 8gb
maxmemory-policy allkeys-lru
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename "dump.rdb"
dir "/data/redis/6400"
slave-serve-stale-data yes
slave-read-only yes
repl-disable-tcp-nodelay no
slave-priority 100
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-entries 512
list-max-ziplist-value 64
set-max-intset-entries 512
zset-max-ziplist-entries 128

3) Create Sentinel configuration file

vim /etc/redis-sentinel6400.conf

daemonize yes
port 26400
dir "/data/redis/redis_sentinels"
pidfile "/var/run/redis/sentinel6400.pid"
logfile "/data/redis/redis_sentinels/sentinel6400.log"
sentinel monitor master6400 100.10.32.54 6400 2
sentinel down-after-milliseconds master6400 6000
sentinel failover-timeout master6400 18000
sentinel client-reconfig-script master6400 /opt/notify_master6400.sh   ## arbitration node does not need this line

For details on Sentinel parameters, see: http://redisdoc.com/topic/sentinel.html

4) Write the floating‑VIP script (used by Sentinel during failover)

vim /opt/notify_master6400.sh

#!/bin/bash
MASTER_IP=$6
LOCAL_IP='100.10.32.54' # change to 100.10.32.55 on the slave
VIP='100.10.32.250'
NETMASK='24'
INTERFACE='eth0'
if [ ${MASTER_IP} = ${LOCAL_IP} ]; then
    /sbin/ip addr add ${VIP}/${NETMASK} dev ${INTERFACE}
    /sbin/arping -q -c 3 -A ${VIP} -I ${INTERFACE}
    exit 0
else
    /sbin/ip addr del ${VIP}/${NETMASK} dev ${INTERFACE}
    exit 0
fi
exit 1

chmod +x /opt/notify_master6400.sh   # make executable

5) Start Redis instances on both nodes redis-server /etc/redis_6400.conf 6) Initialize replication (run on the slave) redis-cli -p 6400 slaveof 10.10.32.54 6400 7) Bind the VIP to the master /sbin/ip addr add 100.10.32.250/24 dev eth0 8) Start Sentinel on all three nodes (master, slave, arbitration)

redis-server /etc/redis-sentinel6400.conf --sentinel

At this point the high‑availability solution is fully deployed.

4. Testing

1) Shut down the master redis-cli -h 100.10.32.54 -p 6400 shutdown 2) Verify the slave has been promoted

redis-cli -h 100.10.32.55 -p 6400 info Replication
# Replication
role:master
connected_slaves:0

3) Check that the VIP has moved to the new master

ip a | grep eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> ...
    inet 100.10.32.55/24 brd 100.10.32.255 scope global eth0
    inet 100.10.32.250/24 scope global secondary eth0

4) Verify Sentinel’s monitoring state

redis-cli -p 26400 info Sentinel
# Sentinel
sentinel_masters:1
master0:name=master6400,status=ok,address=100.10.32.55:6400,slaves=1,sentinels=3

Source: https://blog.51cto.com/navyaijm/1745569