Mastering Cache Architecture: From CDN to Distributed Systems

Cache Overview

Caching stores frequently accessed data closer to the requester, reducing latency and load on backend services in large distributed systems.

Cache Classifications

CDN Cache Content Delivery Networks place cache servers at geographically distributed edge locations. User requests are routed to the nearest healthy node, which serves static assets (images, video, CSS, JavaScript) directly.

Reverse‑Proxy Cache A reverse‑proxy sits in front of application servers, intercepting HTTP requests. Cached responses are returned immediately; otherwise the request is forwarded, the response cached, and then served.

Local Application Cache Runs inside the same process as the application, providing ultra‑fast, in‑process access with no network overhead. Suitable for single‑instance services or clusters where each node maintains its own cache. Drawback: memory duplication and lack of sharing across instances.

Distributed Cache Independent cache services (e.g., Memcached, Redis) shared by multiple applications. Decouples caching from the application process, enabling horizontal scaling, high availability, and centralized cache management.

Popular Cache Implementations

Ehcache Java‑based open‑source cache that can operate in‑process or out‑of‑process, scaling to terabytes. Supports configurable eviction policies, persistence, and integration with frameworks such as Spring and Hibernate.

Guava Cache Part of Google’s Guava library. Provides a lightweight in‑memory cache with size‑based or time‑based eviction, refresh‑after‑write, and automatic removal listeners.

Memcached High‑performance distributed memory object cache. Stores data in a large hash table in RAM, offering simple key‑value access, LRU eviction, and client‑side sharding for horizontal scaling.

Redis In‑memory data store supporting strings, hashes, lists, sets, sorted sets, streams, Lua scripting, replication, clustering, Sentinel for high availability, and multiple eviction policies.

Common Cache Challenges

Data Consistency – Stale copies can cause dirty reads.

Cache Penetration – Repeated queries for non‑existent keys bypass the cache and overload the database.

Cache Avalanche – Simultaneous expiration of many keys generates a traffic surge to the backend.

High Availability – Cache must remain reachable despite node failures or network partitions.

Cache Hotspot – A few hot keys receive massive concurrent requests, stressing a single cache node.

Mitigation Strategies

Data Consistency : Use write‑through or write‑behind patterns, versioned entries, and periodic invalidation.

Cache Penetration : Cache empty results for missing keys and employ Bloom filters to pre‑filter impossible keys.

Cache Avalanche : Stagger TTLs with random offsets, keep a warm‑up cache layer, and pre‑populate critical data.

High Availability : Deploy multiple nodes with replication or sharding; implement client‑side failover and health checks.

Cache Hotspot : Replicate hot keys across several nodes or use consistent hashing with virtual nodes to distribute load.

Industry Case Study – Sina Weibo Feed Cache

Sina Weibo introduced an SSD‑based L2 cache between Redis/Memcached and MySQL. The L2 layer reduced pure‑memory cache cost, mitigated cache‑penetration pressure on the database, and improved overall latency and storage efficiency.

Technical Design Considerations

Multi‑layer cache architectures increase complexity in data consistency, eviction policy coordination, and observability. Designers must balance latency, cost, and reliability while providing clear metrics and alerts for each cache tier.

Key Takeaways

Effective cache architectures combine multiple layers—CDN, reverse‑proxy, local, and distributed caches—to satisfy diverse latency and scalability requirements. Understanding common pitfalls (consistency, penetration, avalanche, availability, hotspot) and applying proven mitigation techniques is essential for building robust, high‑performance systems.