Comprehensive Guide to Caching: Principles, Types, Strategies, and Best Practices

1. Cache Overview

Cache is a memory hash used as a buffer to speed up data access by storing frequently used data closer to the CPU, application, or user, thereby improving read latency and reducing storage costs.

Cache Hit Rate = Reads From Cache / Total Reads

2. When to Introduce Cache

High CPU cost operations (e.g., complex regex, intermediate computation results)

High I/O cost (e.g., busy DB connection pool)

Benefits include faster reads, better scalability, and lower storage cost.

3. Cache Types

Client‑side : HTTP cache, browser cache, app cache (Android, iOS).

Server‑side : CDN cache, reverse‑proxy cache, DB cache, in‑process cache, distributed cache.

Backend developers usually focus on in‑process and distributed caches.

4. Cache Eviction Strategies

Space‑based (size limit)

Capacity‑based (record count)

Time‑based (TTL, TTI)

Common algorithms: FIFO, LRU, LFU. LRU is often chosen for its balance of cost and hit rate.

5. In‑Process Cache Implementations

ConcurrentHashMap (no eviction)

LRUHashMap (extends LinkedHashMap)

Guava Cache (segment‑locked, lazy expiration)

Caffeine (W‑TinyLFU, high hit rate)

Ehcache (two‑level, persistence, clustering)

6. Distributed Cache

Key solutions: Memcached and Redis.

Memcached uses slab allocation, LRU + lazy expiration, no persistence, client‑side sharding.

Redis offers multiple data structures, various eviction policies (volatile‑lru, allkeys‑lru, etc.), persistence (RDB, AOF), clustering, and asynchronous replication.

7. Multi‑Level Cache Architecture

Typical flow: CDN → Reverse‑proxy → Application server → In‑process cache (L1) → Distributed cache (L2) → Database.

Use L1 for hot data, L2 for broader hot set; avoid over‑complicating with too many layers.

8. Common Cache Problems

Cache avalanche – massive miss causing DB overload; mitigated by high availability, multi‑level cache, rate limiting, random TTL.

Cache penetration – requests for non‑existent keys; mitigated by caching nulls or using Bloom filters.

Cache breakdown – hot key expiration spikes; mitigated by distributed locks or proactive refresh.

9. Cache Strategies

Cache warm‑up on startup or scheduled refresh.

Write‑through vs. write‑behind; prefer delete‑then‑update or update‑then‑delete to avoid stale reads.

When strong consistency is required, consider 2PC or Paxos (generally avoided).

10. Summary

Effective caching combines appropriate cache types, eviction policies, and multi‑level designs while handling avalanche, penetration, and breakdown through high availability, random TTL, Bloom filters, locks, and asynchronous refresh.