Cache Optimization Techniques and Design Strategies

Cache can significantly accelerate read/write speed and lower backend load, making it essential for modern applications. This article introduces cache usage techniques and design schemes, covering benefit‑cost analysis, update strategies, granularity control, penetration mitigation, bottomless‑hole optimization, avalanche prevention, and hot‑key rebuild optimization.

1) Cache Benefits and Cost Analysis

The left diagram shows a client directly accessing the storage layer, while the right diagram illustrates a typical architecture with a cache layer in front of the storage layer.

Benefits:

Accelerated read/write: caches are usually in‑memory, providing faster access than storage systems like MySQL.

Reduced backend load: fewer requests and complex queries reach the database.

Costs:

Data inconsistency: a time window exists between cache and storage updates.

Code maintenance: developers must handle both cache and storage logic.

Operations: adding a Redis cluster increases operational overhead.

Typical usage scenarios include heavy computational workloads (e.g., complex MySQL joins) and request‑response acceleration, even for simple queries such as select * from table where id=.

2) Cache Update Strategies

Because cached data can become stale, various update strategies are employed:

LRU/LFU/FIFO eviction : Redis uses the maxmemory-policy setting to evict entries when memory limits are reached.

Timeout eviction : Assign an expiration time to keys; after expiry the data is fetched from the source and re‑cached. Suitable for data that can tolerate short inconsistency periods.

Active update : Immediately refresh the cache after the underlying data changes, often via message queues.

Comparison of the three strategies is illustrated below:

Recommendations:

Low‑consistency workloads: configure max memory and eviction policy.

High‑consistency workloads: combine timeout eviction with active updates.

3) Cache Granularity Control

Choosing the right cache granularity balances space usage, network bandwidth, and code maintainability. A common setup uses Redis as the cache layer and MySQL as the storage layer.

4) Penetration Optimization

Cache penetration occurs when requests query non‑existent data, causing both cache and storage misses. Two common solutions are:

Cache empty objects : Store a placeholder for missing keys to prevent repeated storage hits. Set a short TTL to limit memory waste.

Bloom filter : Pre‑filter keys using a Bloom filter; only keys that may exist are forwarded to the storage layer.

5) Bottomless‑Hole Optimization

When many cache nodes are added, batch operations can become slower due to increased network calls. Optimizations include command‑level tuning, reducing network round‑trips, and using connection pooling or NIO.

Four batch‑operation approaches for Redis Cluster are compared:

Serial commands (n network calls).

Serial I/O with slot‑aware grouping (node‑count network calls).

Parallel I/O using multithreading (still node‑count calls, but latency O(1)).

Hash‑tag routing to force keys onto the same node (single network call).

6) Avalanche Optimization

A cache avalanche happens when the cache becomes unavailable, causing a sudden surge of traffic to the storage layer. Prevention measures include ensuring high availability of the cache (e.g., Redis Sentinel or Cluster), isolating backend resources with circuit breakers and thread pools, and conducting failure‑injection drills.

7) Hot‑Key Rebuild Optimization

When a hot key expires, many concurrent requests may try to rebuild the cache, overwhelming the backend. Two common solutions are:

Mutex lock : Use Redis SETNX (or SET ... NX EX) to allow only one thread to rebuild the cache while others wait.

Never‑expire with logical TTL : Store the data indefinitely but attach a logical expiration; a background thread refreshes the cache when the logical TTL expires.

Mutex lock flow:

Never‑expire flow:

Both methods reduce the risk of cache‑stampede while balancing consistency and performance.

---

For further reading, see the related articles linked at the end of the original post.