Mastering Redis Distributed Locks: From SETNX to RedLock and WatchDog

Background

Local JVM locks such as synchronized or ReentrantLock cannot provide mutual exclusion across multiple processes or machines. A distributed lock that all services can access is required, and Redis is commonly used because of its high throughput and atomic commands.

Level 1 – Simple Mutual Exclusion with SETNX

Core Idea

Redis command SETNX key value creates the key only if it does not exist and returns 1 (lock acquired) or 0 (lock failed). The lock key is typically named resource_lock.

Process Flow

Lock acquisition : Multiple clients execute SETNX resource_lock 1 concurrently.

Success : The client that receives 1 holds the lock.

Failure : Others receive 0 and must retry or give up.

Release : The holder deletes the key with DEL resource_lock.

Problem – Deadlock

If the holder crashes before releasing the key, the lock remains forever because no expiration was set, causing a deadlock.

Level 2 – Adding a TTL (SETNX + EXPIRE)

Non‑Atomic Pitfall

Executing SETNX and EXPIRE as two separate commands is not atomic. A crash between the two calls leaves a key without TTL.

// Wrong: non‑atomic lock acquisition
if (jedis.setnx("lock_key", "1") == 1) {
    // crash may happen here
    jedis.expire("lock_key", 10); // 10‑second TTL
    try {
        // business logic
    } finally {
        jedis.del("lock_key");
    }
}

Atomic Solution – SET with NX and PX

Since Redis 2.6.12 the SET command can combine existence check and expiration atomically.

String result = jedis.set(
    "lock_key",
    "value",   // unique identifier, e.g., UUID
    "NX",      // set only if not exists
    "PX",      // expiration in milliseconds
    10000);     // 10 s TTL
if ("OK".equals(result)) {
    // lock acquired, will expire automatically after 10 s
}

Level 3 – Preventing Accidental Unlock

Mis‑deletion Problem

If a client’s lock expires while its business logic is still running, another client may acquire the lock. When the original client finally executes DEL lock_key, it deletes the new owner’s lock.

Solution – Store a Unique Identifier

Store a UUID (or UUID + thread‑ID) as the lock value. During unlock, compare the stored value with the client’s identifier and delete only if they match.

Lua‑Based Atomic Unlock

if redis.call('get', KEYS[1]) == ARGV[1] then
    return redis.call('del', KEYS[1])
else
    return 0
end

This script guarantees that the key is removed only when the value matches the caller’s UUID.

Level 4 – Automatic Renewal (WatchDog)

Expiration Dilemma

A short TTL may expire while business logic is still executing; a long TTL increases the risk of deadlock if the client crashes.

Redisson WatchDog Mechanism

Redisson starts a background thread that runs every lockWatchdogTimeout/3 (default 10 s). If the client still holds the lock, the thread extends the TTL to the default 30 s via a Lua script. If the client crashes, the thread stops and the lock naturally expires after 30 s.

Level 5 – Redis Cluster Consistency (RedLock)

Problem in Master‑Slave or Cluster Deployments

Replication is asynchronous. If a master crashes before propagating the lock to its slaves, a new master may be elected without the lock, allowing another client to acquire the same key simultaneously.

RedLock Algorithm

Record start timestamp T1.

Attempt SET NX PX on N independent Redis masters (commonly 5) with a short connection timeout.

Compute elapsed time T2‑T1.

Consider the lock successful only if:

If the criteria are not met, release any partial locks.

Discussion

RedLock improves consistency at the cost of performance and still faces clock‑skew risks. For most high‑concurrency internet services (e.g., rate limiting, inventory deduction) a single Redis instance with Redisson’s WatchDog is sufficient. For strict financial transactions, a CP system such as Zookeeper or Etcd is recommended.

Checklist for a Robust Redis Distributed Lock

Atomic mutual exclusion : Use SET … NX PX instead of separate SETNX and EXPIRE.

Unique identifier : Store a UUID (or client ID) as the lock value to prevent accidental deletion.

Safe unlock : Execute the Lua script that checks the identifier and deletes the key atomically.

Automatic renewal : Employ a WatchDog (e.g., Redisson) that extends the TTL while the business logic runs.