Why Redis Distributed Locks Fail in Flash Sales and How to Fix Them
This article examines a real flash‑sale incident where Redis distributed locks expired, causing overselling, analyzes the root causes such as lock safety and non‑atomic stock checks, and presents safer lock implementations, atomic stock operations, and architectural reflections to prevent future failures.
Preface
Using Redis for distributed locks is common. This article analyzes a real incident where a flash‑sale of a scarce product caused overselling due to lock expiration and non‑atomic stock checks, and presents solutions.
Incident Scene
During a "Flying Moutai" flash‑sale with 100 bottles, the system oversold because the distributed lock expired while the request was blocked on user‑service validation, allowing another thread to acquire the lock and release it incorrectly.
Root Causes
No fallback handling for user‑service latency. The gateway delay caused lock timeout.
Distributed lock is unsafe. Lock expiration allows another thread to acquire and later release the original lock.
Non‑atomic stock verification. Get‑and‑compare logic is not atomic, leading to oversell.
Solution
Safer Distributed Lock
Implement lock release based on a unique value and use a Lua script for atomic get‑and‑compare.
public void safedUnLock(String key, String val) {
String luaScript = "local in = ARGV[1] local curr=redis.call('get', KEYS[1]) if in==curr then redis.call('del', KEYS[1]) end return 'OK'";
RedisScript<String> redisScript = RedisScript.of(luaScript);
redisTemplate.execute(redisScript, Collections.singletonList(key), Collections.singleton(val));
}Atomic Stock Check
Leverage Redis' atomic increment operation to decrement stock safely.
Long currStock = redisTemplate.opsForHash().increment("key", "stock", -1);Refactored Business Logic
Introduce a DistributedLocker class, generate a UUID as lock value, acquire lock with timeout, perform atomic stock decrement, and release lock via the Lua script.
public SeckillActivityRequestVO seckillHandle(SeckillActivityRequestVO request) {
SeckillActivityRequestVO response;
String key = "key:" + request.getSeckillId();
String val = UUID.randomUUID().toString();
try {
Boolean lockFlag = distributedLocker.lock(key, val, 10, TimeUnit.SECONDS);
if (!lockFlag) {
// business exception
}
// user activity validation
// atomic stock decrement
Long currStock = stringRedisTemplate.opsForHash().increment(key + ":info", "stock", -1);
if (currStock < 0) {
// out of stock handling
} else {
// generate order, publish event, build response
}
} finally {
distributedLocker.safedUnLock(key, val);
}
return response;
}Deep Reflection
Is a Distributed Lock Necessary?
Redis atomic decrement can prevent oversell, but without a lock the high‑traffic validation may overload downstream services, so a lock can still be useful for traffic shaping.
Lock Selection
RedLock offers higher reliability at the cost of performance; choose based on reliability vs. latency requirements.
Conclusion
Overselling scarce items is a severe incident. Proper lock handling, atomic operations, and thorough performance testing are essential to avoid such failures.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Programmer DD
A tinkering programmer and author of "Spring Cloud Microservices in Action"
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
