How to Prevent Cache Consistency, Concurrency, and Avalanche Issues in High‑Traffic Systems

1. Cache Consistency Issues

When data freshness is critical, the cache must stay consistent with the database, and cache nodes and replicas must also be synchronized to avoid discrepancies.

This relies on expiration and update strategies; typically the cache is proactively updated or the relevant entry removed when the underlying data changes.

2. Cache Concurrency Issues

After cache expiration, requests fall back to the backend database, which under high concurrency can cause many simultaneous DB accesses, leading to a “cache avalanche”.

When a cache key is being updated while many requests read it, consistency problems arise. A common mitigation is to use a lock: acquire a lock during update or DB fetch, release it after completion, allowing other requests to wait briefly and then read from the cache.

3. Cache Penetration Issues

Cache penetration (often called “cache breakdown”) is not merely a cache failure; it occurs when a hot key is repeatedly missed and the corresponding data in the database is empty, causing a flood of unnecessary DB queries.

Common prevention methods include:

Cache empty objects – store an empty collection or a placeholder for null results to stop requests from reaching the DB.

Separate filter handling – intercept keys that may return empty data and handle them specially before reaching the DB.

4. Cache Jitter Issues

Cache jitter (or “cache wobble”) is a milder fault than an avalanche, often caused by node failures; consistent hashing is recommended to mitigate it.

5. Cache Avalanche Phenomenon

A cache avalanche happens when a large number of requests bypass a failing cache and overwhelm the backend database, potentially crashing the entire system.

It can be triggered by concurrency spikes, penetration, jitter, or even malicious attacks. Staggered expiration times can reduce the risk of simultaneous cache expiry.

From an architectural perspective, rate limiting, degradation, circuit breaking, and multi‑level caching can lessen the impact, while thorough pressure testing helps expose issues early.

6. Cache Bottomless Hole Phenomenon

First observed at Facebook, the “bottomless hole” describes a situation where adding more memcached nodes does not improve performance because connection‑frequency overhead dominates.

Sharding (hash or range) distributes data across nodes, but each operation may involve network communication with multiple instances, increasing overhead.

Key mitigation strategies include:

Choosing an appropriate data distribution method (hash vs. range) based on workload.

IO optimization – use connection pools, NIO, etc., to reduce connection costs.

Data access optimization – fetch larger data sets in a single request rather than many small ones.

In most companies this issue is rare.