Using Redis for Order List Management in JD Daojia: Storage Structure, Consistency, Distributed Locks, and Cache Protection

Redis, a high‑performance in‑memory database, is widely used in internet companies. This article describes its application in JD Daojia's order list, covering storage structure, data‑consistency guarantees, distributed locking, and cache‑penetration/avalanche protection.

1. Order list storage structure in Redis – Each user's orders are cached using the user ID as the key and a sorted set ordered by a score composed of the order timestamp (in milliseconds) plus the last three digits of the order number. Only the most recent N orders are kept to save memory.

2. Ensuring consistency between Redis and the database – After a successful DB update, the cache is refreshed. If the cache update fails, two strategies are used:

Retry the cache update up to five times; if still failing, a worker periodically scans the DB and corrects the cache.

After five retries, send an MQ message to trigger an asynchronous cache update, which is the current production approach.

for (int i = 0; i < 5; i++) {
    try {
        // put data into cache
        addOrderListRedis(key, score, orderListVO);
        break;
    } catch (Exception e) {
        log.error("{}IOrderRedisCache.putOrderList2OrderListRedis--->>jdCacheCloud.zAdd exception：", logSid, e);
        if (i == 4) sendUpOrderCacheMQ(orderListVO, logSid); // send MQ after 5 failures
    }
}

3. Distributed lock in Redis – Since Redis 2.6.12, a simple atomic command can create a lock: SET key value EX seconds NX. For earlier versions, a combination of SETNX and EXPIRE (or GETSET) is required. Example implementations are provided:

public boolean getLock(String lockKey, String lockValue){
    if(shardedXCommands.set(lockKey, lockValue, 10, TimeUnit.SECONDS, false)) {
        return true;
    }
    return false;
}

public boolean getLock(String lockKey){
    boolean lock = false;
    while (!lock) {
        String expireTime = String.valueOf(System.currentTimeMillis() + 5000);
        lock = shardedXCommands.setNX(lockKey, expireTime);
        if (lock) return true;
        String oldTimeStr = shardedXCommands.get(lockKey);
        if (oldTimeStr != null && !oldTimeStr.trim().isEmpty()) {
            long oldTime = Long.valueOf(oldTimeStr);
            long now = System.currentTimeMillis();
            if (oldTime < now) {
                String oldTimeStr2 = shardedXCommands.getSet(lockKey, String.valueOf(now + 5000));
                if (oldTimeStr.equals(oldTimeStr2)) {
                    lock = true;
                    break;
                }
            }
        }
        try { Thread.sleep(50); } catch (InterruptedException e) { log.error(e); }
    }
    return lock;
}

The lock is used when an order status changes: acquire lock → read cache → verify status → update cache → release lock.

4. Cache penetration and avalanche protection – When many hot keys expire simultaneously or malicious requests flood the system, the database can be overwhelmed. The solution combines distributed locking with limited lock granularity (128 locks) so that only a bounded number of threads can query the DB after a cache miss.

// Initialize 128 lock objects
public static final String[] LOCKS = new String[128];
static {
    for (int i = 0; i < 128; i++) {
        LOCKS[i] = "lock_" + i;
    }
}

public List<OrderVOList> getOrderVOList(String userId){
    if (orderRedisCache.isOrderListExist(userId)) {
        return getOrderListFromCache(userId);
    }
    int index = userId.hashCode() & (LOCKS.length - 1);
    try {
        orderRedisCache.lock(LOCKS[index]);
        if (orderRedisCache.isOrderListExist(userId)) {
            return getOrderListFromCache(userId);
        }
        List<OrderVOList> list = getOrderListFromDb(userId);
        if (list == null || list.isEmpty()) {
            jdCacheCloud.zAdd(OrderRedisKey.getListKey(userId), 0, null);
        } else {
            jdCacheCloud.zAdd(OrderRedisKey.getListKey(userId), list);
        }
        return list;
    } finally {
        orderRedisCache.unlock(LOCKS[index]);
    }
}

By limiting lock granularity, at most 128 concurrent DB queries can occur, dramatically reducing load spikes.

Summary

Orders are cached per user as a sorted set keyed by timestamp + order‑id suffix, storing only the most recent N orders.

Data consistency is handled by retrying cache updates and falling back to MQ‑driven asynchronous updates.

Redis distributed locks use SET … EX NX after 2.6.12; earlier versions combine SETNX and GETSET.

Cache‑penetration and avalanche mitigation rely on a limited number of hashed locks, ensuring that only a controlled number of threads can hit the database after a miss.

For more technical articles, follow the XinDada Technology public account.