Cache Design and Optimization Strategies in Distributed Systems

In high‑concurrency distributed systems, caching is essential for accelerating reads/writes and reducing backend load, but it introduces costs such as data inconsistency, code maintenance, and operational overhead. When the benefits outweigh these costs, caching should be adopted.

Cache update strategies include:

LRU / LFU / FIFO eviction for limited memory scenarios where data changes rarely.

Timeout eviction (e.g., Redis expire ) which relies on a configurable TTL, suitable for workloads tolerant of eventual consistency.

Active updates that synchronize cache with the data source, offering the highest consistency at the expense of higher development and operational complexity.

Best practices suggest using low‑consistency strategies (1 + 2) for tolerant workloads and combining strategies 2 + 3 for high‑consistency requirements.

Penetration mitigation :

Caching empty objects to block repeated queries for non‑existent keys, with short TTLs and business‑level request filtering.

Employing Bloom filters to quickly test key existence before hitting the database, which can be combined with local caching for minimal memory overhead.

Bottom‑hole (no‑bottom‑hole) optimization addresses the performance degradation caused by batch operations (e.g., mget ) spreading across many nodes. Solutions range from serial execution to parallel I/O and hash‑tag routing that forces related keys onto the same node, reducing network round‑trips.

Avalanche prevention involves ensuring cache high availability (master‑slave, Redis Sentinel), using isolation components for rate‑limiting and fallback (e.g., Hystrix), and isolating projects to contain failures.

Hot‑key rebuild mitigation includes:

Mutex locks so only one thread rebuilds a hot key while others wait or serve stale data.

Never‑expire caches refreshed by scheduled jobs or proactive updates.

Backend rate‑limiting to throttle rebuild requests, especially when hot keys are unknown in advance.

The article concludes with a summary of these cache design insights and invites readers to follow or join the author’s technical community.