Why Redis Distributed Locks Fail and How to Prevent Flash‑Sale Over‑selling

Incident Overview

Using Redis for distributed locks is common, but a recent flash‑sale of 100 bottles of a scarce product resulted in a P0 incident where the stock was oversold by 100 due to lock expiration and non‑atomic stock checks.

Root Causes

No fault‑tolerance for downstream services; user‑service latency caused lock to expire.

Distributed lock implementation was unsafe – lock could be released by another thread after expiration.

Stock verification was non‑atomic (get‑then‑compare), leading to race conditions.

Solution

Safer Distributed Lock

Implement lock release with a Lua script that deletes the key only when the stored value matches the owner’s UUID.

public void safedUnLock(String key, String val) {
    String luaScript = "local in = ARGV[1] local curr=redis.call('get', KEYS[1]) if in==curr then redis.call('del', KEYS[1]) end return 'OK'";
    RedisScript<String> redisScript = RedisScript.of(luaScript);
    redisTemplate.execute(redisScript, Collections.singletonList(key), Collections.singleton(val));
}

Atomic Stock Decrement

Use Redis’s atomic increment with a negative value to reduce stock, eliminating the need for a separate compare step.

Long currStock = redisTemplate.opsForHash().increment("key", "stock", -1);

Refactored Business Logic

Introduce a DistributedLocker class, generate a unique lock value, acquire the lock, perform atomic stock decrement, and release the lock with the Lua script.

public SeckillActivityRequestVO seckillHandle(SeckillActivityRequestVO request) {
    String key = "key:" + request.getSeckillId();
    String val = UUID.randomUUID().toString();
    try {
        Boolean lockFlag = distributedLocker.lock(key, val, 10, TimeUnit.SECONDS);
        if (!lockFlag) { /* handle lock failure */ }
        Long currStock = stringRedisTemplate.opsForHash()
            .increment(key + ":info", "stock", -1);
        if (currStock < 0) { /* out of stock */ }
        else { /* create order */ }
    } finally {
        distributedLocker.safedUnLock(key, val);
    }
    return response;
}

Further Considerations

Even with atomic stock decrement, a lock can still protect downstream services from overload. RedLock offers higher reliability at the cost of performance; choose based on reliability requirements.

Additional optimizations include sharding stock across servers and using in‑memory caches with hash‑based routing to further reduce contention.

Conclusion

Overselling scarce items is a severe incident; thorough analysis revealed unsafe lock usage and non‑atomic stock checks. Implementing a Lua‑based unlock and leveraging Redis’s atomic operations resolved the issue and improved both safety and performance.