Mastering Cache Pitfalls: Penetration, Avalanche, and Hotspot Solutions

Typical Cache Use Cases

In the following two situations, optimizing the storage system cannot effectively improve performance.

1. Data requiring complex computation

For example, counting online users via a database each time puts heavy load on the DB when the display volume is large.

2. Read‑heavy, write‑light data

For instance, a celebrity’s post may be viewed by tens of millions; each view triggering a SELECT would overwhelm the database.

Cache reduces storage pressure by keeping repeatedly used data in memory for multiple accesses.

Design Considerations

While cache relieves storage load, it adds architectural complexity. Pay attention to the following three issues:

Cache penetration

Cache avalanche

Cache hotspot

Cache Penetration

Occurs when the business system cannot find data in the cache and must query the storage system again.

Two typical cases:

Data does not exist in storage.

If the requested data truly does not exist, each request still hits both cache and storage, allowing malicious actors to cause heavy load. A common mitigation is to store a default placeholder in the cache.

Generating cache data is time‑consuming or resource‑intensive.

When cached data expires, regenerating it can overload the storage system, especially under crawler traffic. Possible mitigations include identifying and blocking crawlers (with SEO impact) and setting up monitoring to handle spikes.

Cache Avalanche

When cached entries expire simultaneously, many requests miss the cache and flood the storage system, potentially causing a cascade failure.

Solutions:

Update lock – protect cache refresh with a lock so only one thread regenerates the cache (use distributed locks for large clusters).

Background refresh – set a long TTL and let a background thread periodically refresh the cache; if memory is low and entries are evicted, notify the background thread to update.

This approach also supports cache pre‑warming after system startup.

Cache Hotspot

Some cache entries become hot spots, receiving a burst of high‑frequency accesses (e.g., a viral post). Even a fast cache server can become a bottleneck.

Solution: replicate the cache across multiple nodes and distribute requests. Ensure replicas have staggered expiration times to avoid simultaneous expiry.

Content compiled from “Learning Architecture from Scratch”.