Mastering Cache Design: 7 Common Pitfalls and Proven Solutions

Cache design is a long‑standing topic; early systems used memcache, while modern applications prefer redis. Introducing a Redis client (e.g., RedisTemplate) as a bean makes initialization straightforward.

For low‑concurrency scenarios the cache design may be simple, but billion‑scale systems require careful planning.

Seven Classic Cache Problems

1. Cache Stampede (集中失效)

When many keys expire simultaneously, all requests fall back to the database, causing a sudden surge in load. The solution is to add a random component to the expiration time (base + random) so keys expire gradually.

2. Cache Penetration

Requests for non‑existent data miss both cache and DB, repeatedly hitting the database. Solutions include storing a special empty value for missing keys or using a BloomFilter to filter out invalid keys before DB access.

3. Cache Avalanche

If a subset of cache nodes fails, the whole cache layer can become unavailable. Mitigation strategies are real‑time monitoring with automatic failover and deploying multiple replicas across different racks.

4. Cache Hotspot

Sudden spikes on a single hot key overload a cache node. Detect hot keys (e.g., with Spark streaming), split the key into numbered shards (key#01, key#02, …) and distribute requests across multiple nodes.

5. Large Keys

Oversized values cause timeouts and network congestion. Solutions: set a size threshold and compress values, use pooling techniques, split large keys into smaller ones, and assign reasonable TTLs.

6. Data Consistency

Since cache is a transient store, data must stay consistent with the database. Approaches include retrying failed cache updates and pushing failed keys to a message queue for asynchronous compensation, or using short TTLs with self‑healing reloads.

7. Concurrent Pre‑warming

When cached data expires, many threads may simultaneously query the DB, overwhelming it. Use a global lock so only one thread fetches from the DB and repopulates the cache, or maintain multiple backup copies to serve requests during recovery.

Design a cache governance dashboard to monitor SLA and dynamically scale hot keys.

Conclusion

Cache design offers many techniques, but the core goal remains: maximize cache hits while preserving data consistency.