Redis Distributed Lock Failure in a Flash‑Sale Scenario and Safer Implementation Strategies

Introduction Using Redis for distributed locking is common, but this article examines a real‑world incident where the lock’s expiration caused a flash‑sale of a scarce product to oversell, leading to a P0‑level outage.

Incident Overview During a limited‑stock "Flying Maotai" promotion (100 bottles), the system sold more than available because the Redis lock expired while the request was still processing, allowing subsequent requests to acquire the same lock and decrement stock incorrectly.

Root Causes 1. High load on the user‑service caused request latency beyond the lock’s 10‑second TTL. 2. The lock was released after expiration, unintentionally freeing a lock held by another thread. 3. Stock verification used a non‑atomic "get‑and‑compare" pattern, leading to race conditions.

Analysis The combination of delayed user validation, lock expiration, and non‑atomic stock checks created a feedback loop that repeatedly broke the lock’s safety guarantees, resulting in overselling.

Solution – Safer Distributed Lock Implement a lock that ties the lock’s value to a unique identifier and releases it only if the identifier matches, using a Lua script for atomicity:

public void safedUnLock(String key, String val) {
    String luaScript = "local in = ARGV[1] local curr=redis.call('get', KEYS[1]) if in==curr then redis.call('del', KEYS[1]) end return 'OK'";
    RedisScript
redisScript = RedisScript.of(luaScript);
    redisTemplate.execute(redisScript, Collections.singletonList(key), Collections.singleton(val));
}

Solution – Atomic Stock Decrement Leverage Redis’s atomic increment operation to adjust stock safely without a separate lock:

Long currStock = redisTemplate.opsForHash().increment("key", "stock", -1);

Improved Business Logic The revised seckillHandle method generates a UUID as the lock value, acquires the safer lock, performs atomic stock decrement, and releases the lock via the Lua script:

public SeckillActivityRequestVO seckillHandle(SeckillActivityRequestVO request) {
    SeckillActivityRequestVO response;
    String key = "key:" + request.getSeckillId();
    String val = UUID.randomUUID().toString();
    try {
        Boolean lockFlag = distributedLocker.lock(key, val, 10, TimeUnit.SECONDS);
        if (!lockFlag) { /* handle lock failure */ }
        Long currStock = stringRedisTemplate.opsForHash().increment(key + ":info", "stock", -1);
        if (currStock < 0) {
            log.error("[抢购下单] 无库存");
            // handle out‑of‑stock
        } else {
            // generate order, publish event, build response
        }
    } finally {
        distributedLocker.safedUnLock(key, val);
    }
    return response;
}

Further Considerations Even with a safer lock, the lock adds latency; using Redis’s atomic decrement alone can avoid overselling, but may increase pressure on downstream services. Alternative designs include sharding stock across servers with hash‑based routing and in‑memory counters for higher throughput.

Conclusion The incident demonstrates that seemingly safe distributed‑lock implementations can become failure points under high concurrency. Combining atomic Redis operations with a robust lock release mechanism mitigates overselling while maintaining system stability.