Mastering Backend Caching: Strategies, Types, Eviction & Common Pitfalls

1. Cache Strategies

Read‑through cache automatically loads missing data from the data source and stores it for future requests. Cache‑aside (lazy) cache returns empty on miss, and the application manually fetches the data and writes it back to the cache.

2. Cache Types

2.1 Local Cache

Lives in the same process heap, offering fast reads but limited space, no cross‑instance consistency, and no persistence.

Advantages: simple, no external dependency, fast reads.

Disadvantages: limited capacity, inconsistent across instances, volatile.

2.2 Distributed Cache

External service shared among multiple services, providing abundant storage and a consistent view.

Advantages: ample space, consistent data, persistence (e.g., Redis).

Disadvantages: requires deployment and operational overhead.

Common Services

Redis – supports many data structures, persistence, clustering, multi‑threaded I/O in 6.0.

Memcached – simple key‑value store, no persistence, client‑side sharding, non‑blocking I/O.

3. Eviction Policies

FIFO – first‑in‑first‑out, removes the oldest entries.

LRU – least‑recently‑used, removes entries not accessed recently; often implemented with a doubly‑linked list and hash map.

LFU – least‑frequently‑used, removes entries with the lowest access count; can be implemented with a min‑heap and hash map.

4. Common Cache Problems

4.1 Consistency

Stale data appears when the source updates but the cache does not. Solutions include setting appropriate TTLs (expire‑then‑refresh) or actively updating the cache after source changes.

4.2 Cache Avalanche

Massive simultaneous expirations cause a surge of traffic to the data source. Mitigation: stagger TTLs, use high‑availability cache clusters, and ensure sufficient capacity.

4.3 Cache Penetration

Requests for non‑existent keys bypass the cache and repeatedly hit the source. Mitigation: validate request parameters and cache null placeholders with a short TTL.

4.4 Cache Breakdown

When a hot key expires, many concurrent requests hit the source, overwhelming it. Mitigation: use mutex or distributed lock for the back‑fill operation, keep hot keys permanently cached, or pre‑warm the cache before expiry.

5. Summary

Understanding and correctly applying cache strategies, types, eviction algorithms, and mitigation techniques is essential for improving backend performance, stability, and user experience.