Mastering Redis: From Basics to Advanced Clustering and Distributed Locks

Redis Basics

If you are not familiar with Redis, you can first read this article https://www.cnblogs.com/wugongzi/p/12841273.html which introduces what Redis is and how to use it.

Redis Pipeline

Normally we use Redis by sending a command, waiting for the round‑trip time (RTT), and then receiving the result. When multiple commands need to be executed, this incurs N RTTs and N network transmissions, which is inefficient.

Redis pipeline solves this by allowing the client to send multiple commands without waiting for replies and then read all responses in a single step, reducing RTT and improving efficiency.

Important Note : When using pipeline, the server queues replies, which can consume a lot of memory. For large batches (e.g., 10,000 commands), it is advisable to split them into reasonable chunks, read the replies, and then send the next batch.

Redis Publish/Subscribe

Publish/subscribe is a messaging pattern where the publisher sends messages and subscribers receive them.

As shown in the diagram, a channel can have multiple subscribers and multiple publishers. Any publisher can send a message to the channel, and all subscribers will receive it.

Publish/Subscribe Demo

I started four Redis clients on the server: two subscribers and two publishers. The subscribers listen on channel01.

Step 1: Publisher 1 sends "wugongzi" to channel01.

Subscriber 1 receives the message:

Subscriber 2 receives the message:

Step 2: Publisher 2 sends "hello-redis" to the channel.

Subscriber 1 receives the message:

Subscriber 2 receives the message:

Redis also supports subscribing to multiple channels.

Redis Expiration Strategy

Setting Expiration Time

Redis can set an expiration time for each key. When the time expires, the key is automatically deleted.

In production, it is recommended to set an expiration for every key; otherwise, memory will fill with cold data.

Command to set expiration:

EXPIRE key seconds

Expiration Deletion Policies

Redis keys can be expired passively (when accessed) or actively (periodic scanning).

Passive: When a client accesses an expired key, Redis deletes it.

Active: Redis runs a background task every second that samples 20 random keys, deletes expired ones, and repeats if more than 25% of keys are expired.

Redis Transactions

Basic Usage

Redis transactions allow multiple commands to be executed atomically.

All commands are serialized and executed without interruption.

The transaction is atomic: either all commands are executed or none.

Transactions are implemented with MULTI, EXEC, DISCARD, and WATCH commands.

Transaction Errors

Two error cases exist:

Error before committing (e.g., a syntax error). The transaction fails and none of the commands are queued.

Error after committing (e.g., a runtime error). Redis does not roll back; the commands already queued are still executed.

Redis transactions do not support rollback.

Why No Rollback

Unlike relational databases, Redis treats command errors as programming bugs that should be caught during development, not at runtime.

Errors are usually due to wrong syntax or wrong key type.

Keeping the implementation simple and fast is a design goal.

Discarding a Transaction

When DISCARD is called, the queued commands are cleared and the client exits the transaction state.

WATCH Command

WATCH

makes EXEC conditional: the transaction only executes if none of the watched keys were modified.

Redis Scripts and Transactions

From Redis 2.6, Lua scripts are atomic; they can be used to implement transactional logic.

Redis Persistence

Why Persistence Is Needed

Redis stores data in memory for high performance. Without persistence, a server crash would lose all data.

Persistence Overview

Redis provides two persistence mechanisms: RDB (snapshot) and AOF (append‑only file).

RDB creates point‑in‑time snapshots.

AOF logs every write operation and can rewrite the file to keep it compact.

RDB

RDB snapshots are triggered by: SAVE (blocking, not recommended for production) BGSAVE (non‑blocking, preferred)

Configured save rules in the config file (e.g., save 900 1)

RDB works by forking a child process that writes the snapshot, allowing the parent to continue serving requests.

RDB Advantages

Compact file suitable for backups.

Minimal impact on the main process.

Faster recovery for large datasets compared to AOF.

RDB Disadvantages

Data loss between snapshots.

AOF

AOF records every write command. On restart, Redis replays the commands to rebuild the dataset.

AOF configuration example:

# Enable AOF
aappendonly yes
appendfilename "appendonly.aof"
# Sync strategy
appendfsync everysec
# Rewrite settings
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

AOF writes are buffered and flushed to disk according to appendfsync settings.

AOF Rewrite : Over time the AOF file grows; Redis rewrites it by keeping only the latest state of each key.

-- Before rewrite
set k1 20
set k2 40
set k1 35
set k3 34
set k2 19

-- After rewrite
set k1 35
set k3 34
set k2 19

Mixed Persistence (Redis 4.0+): Uses aof-use-rdb-preamble to prepend an RDB snapshot to the AOF file, combining benefits of both.

AOF Advantages

Better durability (less data loss).

Mixed persistence reduces file size and speeds up loading.

AOF Disadvantages

Larger file size compared to RDB.

Potentially slower writes depending on sync policy.

Recovery can be slower than RDB.

Redis Distributed Lock

Introduction

Local locks (e.g., synchronized or Lock) work only on a single JVM. In a distributed environment, they become ineffective, so Redis‑based distributed locks are used for scenarios like flash sales and seckill.

Lock Characteristics

Mutual exclusion: only one client can hold the lock at a time.

Safety: only the client that acquired the lock can release it.

High performance and low latency.

Automatic expiration to avoid deadlocks.

Re‑entrancy (optional).

Solution 1: SETNX

setnx key value

If the command returns 1, the lock is acquired; otherwise it fails.

Solution 2: SETNX + EXPIRE

if (setnx(k1, v1) == 1) {
    expire(k1, 10);
    try { /* business logic */ } finally { del(k1); }
}

This approach is not atomic; a crash between SETNX and EXPIRE can leave a lock without expiration.

Solution 3: Lua Script (Atomic SETNX + EXPIRE )

if redis.call('setnx', KEYS[1], ARGV[1]) == 1 then
    redis.call('expire', KEYS[1], ARGV[2])
else
    return 0
end

Solution 4: SET key value NX EX seconds with Value Verification

Store a random value as the lock value. When releasing, compare the stored value with the original random value to ensure only the owner releases the lock.

if (jedis.set(resource, randomValue, "NX", "EX", 100) == 1) {
    try { /* business */ } finally {
        if (randomValue.equals(jedis.get(resource))) {
            jedis.del(resource);
        }
    }
}

Lua script for safe release:

if redis.call('get', KEYS[1]) == ARGV[1] then
    return redis.call('del', KEYS[1])
else
    return 0
end

Redisson Framework

Redisson monitors the lock’s remaining TTL and automatically extends it while the protected method is still executing, preventing premature expiration.

Redlock Algorithm (Cluster Mode)

For a Redis cluster, the Redlock algorithm obtains locks on a majority of nodes (e.g., 3 out of 5) with a short network timeout. The lock is considered acquired only if it succeeds on enough nodes and the total elapsed time is less than the lock’s TTL.

Redis Cluster

Three Cluster Modes

Master‑Slave replication

Sentinel mode

Cluster mode

Master‑Slave Replication

One master handles writes; multiple slaves handle reads. The master asynchronously replicates data to slaves.

Advantages:

Read‑write separation.

Slaves can also serve other slaves, reducing master load.

Non‑blocking replication.

Disadvantages:

No automatic failover; if the master crashes, writes are unavailable.

Manual recovery required.

Storage limited by a single master’s memory.

Sentinel Mode

Built on top of master‑slave replication, Sentinel monitors the cluster, performs automatic failover, and provides clients with the current master address.

Functions:

Monitoring: checks health of master and slaves.

Automatic failover: promotes a slave to master when needed.

Configuration provider: clients query Sentinel for the master.

Notification: informs clients of topology changes.

Down detection includes subjective down (single sentinel) and objective down (majority of sentinels).

Leader election follows a Raft‑like majority rule; the elected sentinel selects a new master based on priority and replication offset.

Cluster Mode

Redis Cluster distributes data across multiple nodes using 16,384 hash slots. Each key’s CRC16 modulo 16384 determines its slot, and each node owns a subset of slots.

Example distribution for three nodes:

Node A: slots 0‑5500

Node B: slots 5501‑11000

Node C: slots 11001‑16383

Adding or removing nodes involves moving hash slots without downtime.

Cluster Practical Setup

Configure six instances (three masters, three slaves) with cluster-enabled yes, unique cluster-config-file, and cluster-node-timeout 5000. Start each instance, then create the cluster:

redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006 --cluster-replicas 1

After creation, the cluster has three masters each with a slave, and hash slots are evenly distributed.

Data verification shows keys are stored on the appropriate master based on slot calculation.

Redis Cache Problems

Cache Penetration

Cache Breakdown

Cache Avalanche

Cache Pollution

Cache Penetration

Occurs when requests for non‑existent data bypass the cache and hit the database repeatedly (e.g., malicious queries). Solutions:

Gateway validation, blacklists, rate limiting.

Cache empty results with a short TTL (e.g., 60 s).

Use Bloom filters to filter out obviously invalid keys.

Cache Breakdown

When a hot key expires, many concurrent requests hit the database simultaneously. Solutions:

Interface rate limiting and circuit breaking.

Locking: the first request loads from DB and populates the cache, others wait for the lock.

Cache Avalanche

Massive expiration of many keys at once overwhelms the database. Solutions:

Stagger expiration times to avoid simultaneous expiry.

Distribute hot data across multiple cache nodes.

Cache Pollution

Old keys without expiration accumulate, consuming memory. Solutions:

Set expiration for all cache entries.

Configure an LRU (least‑recently‑used) eviction policy.

Article originally from https://www.cnblogs.com/wugongzi/p/16827473.html

End

Article finished

not clicked?