Common Cache Problems and Their Mitigation Strategies

1. Cache Consistency Issue

When data freshness is critical, the cache must stay consistent with the database and among cache nodes and replicas; otherwise discrepancies arise. This relies on proper expiration and update policies, typically updating or evicting the cache immediately after data changes.

2. Cache Concurrency Issue

After cache expiration, requests fall back to the backend database, which is reasonable, but under high concurrency many requests may simultaneously hit the database, causing a massive load or "avalanche" effect.

Additionally, when a cache key is being updated, a surge of requests may also target it, leading to consistency problems. A common solution is to use a lock: acquire a lock before updating or loading from the database, release it after completion, and let other requests wait briefly before reading the refreshed cache.

3. Cache Penetration Issue

Cache penetration (also called "breakthrough") occurs when a non‑existent key is repeatedly requested; the cache misses and the system queries the backend database each time, causing unnecessary load, especially when the underlying data is empty.

Common mitigation methods:

1) Cache empty objects – store a placeholder for empty results (e.g., an empty list or a special flag) to prevent repeated database hits.

2) Dedicated filtering – intercept requests for keys known to have empty data and handle them centrally, avoiding backend queries.

4. Cache Jitter (Oscillation) Issue

Cache jitter, a milder form of avalanche, occurs when cache nodes fail, causing temporary performance degradation. Using consistent hashing can distribute load more evenly and mitigate this effect.

5. Cache Avalanche Phenomenon

A cache avalanche happens when many cache entries expire simultaneously, flooding the backend database and potentially crashing the entire system.

Causes include the previously mentioned concurrency, penetration, jitter problems, or malicious attacks. Staggered expiration times, rate limiting, degradation, circuit breaking, and multi‑level caching are recommended countermeasures, along with thorough stress testing.

6. Cache "Bottomless Pit" Phenomenon

This issue was observed by Facebook engineers when scaling memcached to thousands of nodes; connection frequency caused performance degradation that persisted even after adding more nodes.

Modern stacks (databases, caches, NoSQL, search middleware) use sharding to meet high performance, concurrency, availability, and scalability requirements. However, each operation may involve network communication with different nodes, increasing overhead as node count grows.

Mitigation strategies include:

1) Data distribution – choose hash‑based or range‑based sharding according to workload characteristics.

2) I/O optimization – employ connection pools, NIO, and other techniques to reduce connection overhead.

3) Access pattern – fetch larger data sets in fewer calls rather than many small calls to lower network I/O.

In most companies, the bottomless‑pit scenario is rare.

Java Group Leader

Focused on sharing Java expertise

Scan the QR code above for more Java content