Mastering Redis: Transactions, Pipelines, Pub/Sub, Replication, Sentinel & Cluster

Redis Transactions

Redis transactions are a set of commands that are executed atomically; either all commands run or none. Redis implements this with MULTI, DISCARD, EXEC and WATCH. MULTI starts a transaction, EXEC executes it and returns results, DISCARD aborts it, and WATCH monitors keys for changes to abort the transaction if they are modified by another client.

127.0.0.1:6379> multi
OK
127.0.0.1:6379> set key 1
QUEUED
127.0.0.1:6379> get key
QUEUED
127.0.0.1:6379> exec
1) OK
2) "1"

127.0.0.1:6379> multi
OK
127.0.0.1:6379> set key1 1
QUEUED
127.0.0.1:6379> discard
OK
127.0.0.1:6379> get key1
(nil)

If a watched key is modified by another client, EXEC will return (nil) and the transaction is not applied.

127.0.0.1:6379> set key 1
OK
127.0.0.1:6379> watch key
OK
127.0.0.1:6379> multi
OK
127.0.0.1:6379> incr key
QUEUED
127.0.0.1:6379> incr key  # client 2
(integer) 2
127.0.0.1:6379> exec
(nil)
127.0.0.1:6379> get key
"2"

Because a transaction holds the server exclusively while executing, it is advisable to keep the number of commands small to avoid blocking the server.

Redis Pipeline

Redis uses a TCP client‑server model similar to HTTP. Normally a client sends a command and blocks waiting for the reply. When network latency is high this adds overhead. The pipeline feature lets a client send many commands without waiting for each response, reducing round‑trip time.

from redis import Redis
import time
conn=Redis(host="60.205.177.100",port="6379")

def usepipline():
    start_time=time.time()
    pipline=conn.pipeline()
    for i in range(300):
        pipline.incr("key")
    pipline.execute()
    print("usepipline:",time.time()-start_time)

def withoutpipline():
    start_time=time.time()
    for i in range(300):
        conn.incr("key1")
    print("withoutpipline:",time.time()-start_time)

usepipline()
withoutpipline()

Result:

usepipline: 1.2412519454956055
withoutpipline: 7.2261717319488525

The pipeline dramatically speeds up bulk operations.

Redis Publish/Subscribe

Redis implements a publish/subscribe model with PUBLISH and SUBSCRIBE commands. Publishers send messages to channels, and any number of subscribers can listen to those channels. This decoupling improves scalability and allows dynamic network topologies.

Commands

Publish a message

publish channel message

Subscribe to a channel

subscribe channel [channel ...]

Unsubscribe

UNSUBSCRIBE [channel [channel ...]]

Pattern subscribe (PSUBSCRIBE)

Allows subscribing to multiple channels using glob patterns.

PSUBSCRIBE pattern [pattern ...]

Pattern unsubscribe

PUNSUBSCRIBE [pattern [pattern ...]]

Python demo

redis_pub.py

from redis import Redis
import time
conn=Redis(host="60.205.177.100",port="6379")
def publish():
    while True:
        conn.publish("CCTV3","test")

redis_sub.py

from redis import Redis
import time
conn=Redis(host="60.205.177.100",port="6379")
def subscribe():
    subscribe=conn.pubsub()
    subscribe.subscribe('CCTV3')
    message=subscribe.parse_response()
    print(message)

Redis Replication

Beyond AOF and RDB persistence, replication creates multiple copies of data to avoid single‑point failure.

Key features

Asynchronous replication with server‑side progress acknowledgment once per second.

One master can have many slaves; slaves can also have slaves, forming cascaded topologies.

Replication is non‑blocking on the master; slaves may serve stale data depending on the replica-serve-stale-data setting.

Replication can be used for redundancy or to offload read‑only queries to slaves.

Even with replication, the master should keep persistence enabled because a restarted master starts with an empty dataset.

The master‑slave model still has a single‑point risk: if the master fails, writes are impossible until a failover occurs.

Synchronization modes

Full sync

The master sends an RDB snapshot to the slave. While the snapshot is transferred, new writes are buffered. After the slave loads the snapshot, it may rewrite AOF if enabled, and then the buffered writes are applied.

Partial sync

If the connection drops, the slave sends PSYNC with the last offset and master ID. If the master still has the offset in its buffer, it continues from there; otherwise a full sync is performed.

psync masterID offset

Configuration

When the master requires a password, slaves must specify masterauth in their config. masterauth 123456 Replication can be set via config file ( replicaof or legacy slaveof) or at runtime with the same commands; the latter does not survive a restart.

replicaof masterip port   # Redis 5.0+
# or
slaveof masterip port

Redis Sentinel

In a master‑slave setup, manual failover is required when the master crashes. Redis Sentinel (available since 2.8) monitors masters and slaves via heartbeats and automatically promotes a slave to master when needed, reconfiguring the remaining replicas.

Sentinel functions

Monitoring – continuously checks that masters and slaves are operating as expected.

Notification – informs clients via API when a node is down.

Automatic failover – promotes a slave to master and reconfigures other slaves.

Configuration provider – clients query sentinels for the current master address.

Setup example

Environment

192.168.179.131:6379 master
192.168.179.132:6379 slave
192.168.179.134:6379 slave
192.168.179.131:26379 sentinel
192.168.179.132:26379 sentinel
192.168.179.134:26379 sentinel

Master/Slave config

# Master
port 6379
daemonize yes
logfile /var/log/redis_6379.log
dbfilename dump.rdb

# Slave 1
port 6379
daemonize yes
logfile /var/log/redis_6379.log
dbfilename dump.rdb
replicaof 192.168.179.131 6379

# Slave 2
port 6379
daemonize yes
logfile /var/log/redis_6379.log
dbfilename dump.rdb
replicaof 192.168.179.131 6379

Sentinel config (same on all three sentinels)

port 26379
daemonize yes
sentinel monitor mymaster 192.168.179.131 6379 2

Start sentinels

./src/redis-sentinel sentinel.conf
# or
./src/redis-server sentinel.conf --sentinel

After stopping the original master, the remaining slaves elect a new master automatically.

Sentinel management commands

info sentinel

– basic info about monitored masters. sentinel masters – list all monitored masters. sentinel master <name> – detailed info about a master. sentinel slaves <name> – details of a master's slaves. sentinel sentinels <name> – details of sentinel nodes. sentinel get-master-addr-by-name mymaster – returns master IP and port. sentinel is-master-down-by-addr – checks if a master is down. sentinel failover <name> – forces a failover. sentinel ckquorum <name> – checks quorum. sentinel flushconfig – forces config write. sentinel remove <name> – stops monitoring a master.

Redis Cluster

When a single master cannot handle large data volumes, Redis Cluster provides a distributed, fault‑tolerant architecture. It offers a subset of single‑node features, uses no central proxy, and aims for linear scalability. Each node can be a master or a replica; replicas serve as failover candidates.

Node responsibilities

Store key‑value pairs.

Maintain cluster state, including key‑to‑node mapping.

Discover other nodes, detect failures, and elect new masters.

Key distribution

The keyspace is divided into 16384 slots. When a node is added or removed, slots are migrated accordingly. A master can have many replicas for redundancy.

Cluster node attributes

IP address and TCP port.

Flags indicating role.

Assigned hash slots.

Last ping and pong timestamps.

Fail time, number of replicas, and master ID (if replica).

Data consistency

Redis Cluster provides eventual consistency; writes may be lost if a master fails before replicating to its slaves, or during network partitions.

Failure detection

A node marks another as PFAIL if it does not answer PING within the timeout.

Nodes gossip PFAIL/FAIL status to others.

If a majority consider a node PFAIL, it becomes FAIL.

FAIL status is broadcast to the whole cluster.

Cluster setup

Enable cluster in config

cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-node-timeout 15000

Create cluster and assign slots

./redis-cli --cluster create 192.168.179.131:6379 192.168.179.131:6380 192.168.179.134:6379 192.168.179.134:6380 192.168.179.132:6379 192.168.179.132:6380 --cluster-replicas 1

Check cluster status

./redis-cli --cluster check 192.168.179.132:6379

Validate data placement

./redis-cli -c -h 192.168.179.132 -p 6380
192.168.179.132:6380> set key 1
-> Redirected to slot [12539] located at 192.168.179.132:6379
OK
192.168.179.134:6379> get key
-> Redirected to slot [12539] located at 192.168.179.132:6379
"1"