How to Prevent Cache Breakdown, Expiration, and Hot Key Issues in Distributed Systems

Background

Distributed caches such as Redis, Memcached, or Tair are used in read‑heavy, write‑light services to reduce database load and support high concurrency.

Typical Cache Issues

Cache breakdown (miss storm)

When a key does not exist in the underlying database, every request falls through to the DB. In high‑traffic scenarios this can overwhelm the database.

Cache expiration surge

If many keys expire at the same moment, especially hot keys, a burst of DB queries is generated.

Hot key concentration

A single hot key stored on one cache node directs all traffic to that node, creating a bottleneck that cannot be solved by simply adding more nodes.

Mitigation Techniques

Preventing cache breakdown

Store a sentinel value (e.g., the string "NULL") for missing data with a short TTL (e.g., 30 seconds). Subsequent requests hit the cache and avoid DB access until the sentinel expires.

Mitigating expiration spikes

Stagger TTLs: assign different expiration times to related keys (e.g., add a random offset of 0‑300 seconds) so they do not expire simultaneously.

Use a distributed lock on cache miss:

if (!cache.contains(key)) {
    if (lock.tryLock(key)) {
        value = db.query(key);
        cache.set(key, value, ttl);
        lock.unlock(key);
    } else {
        // wait briefly then retry reading from cache
        Thread.sleep(50);
        return cache.get(key);
    }
}

Only the process that acquires the lock reads the DB and repopulates the cache; others wait or retry.

Hot‑key mitigation

Client‑side hot‑key cache: keep the hot value locally with its own TTL and check the local cache before contacting the distributed cache.

Key sharding: split the hot key into N sub‑keys (e.g., key_0 … key_N‑1) and store each sub‑key on a different cache node. When reading, select a sub‑key via hash(key) % N and retrieve the value.