Mastering Cache Strategies: From CDN to Distributed Systems

Cache Overview

The article introduces the theory of caching in large distributed systems, enumerates common cache components, and outlines typical application scenarios.

Cache Classification

Caching is divided into four major categories, each with its own principles and use cases.

CDN Cache

Content Delivery Networks (CDN) deploy numerous cache servers close to end users. Global load‑balancing directs requests to the nearest healthy cache node, which serves static resources such as images and videos directly, reducing latency and origin‑server load.

Reverse‑Proxy Cache

Placed in the application‑server data center, a reverse‑proxy intercepts all web requests. It caches small static files (CSS, JS, images) and serves them directly when available, otherwise forwards the request to the origin server and caches the response for future hits.

Local Application Cache

Implemented inside the application process, local caches (e.g., Ehcache, Guava Cache) provide ultra‑fast access without network overhead. They are ideal for single‑instance services or scenarios where nodes do not need to share cached data, but they suffer from memory waste and lack of cross‑node sharing.

Distributed Cache

A separate cache service that can be shared across multiple applications, providing scalability and high availability. The article highlights two dominant open‑source solutions: Memcached and Redis.

Memcached

Memcached is a high‑performance, distributed in‑memory object cache that stores data in a large hash table. It is commonly used to cache static assets, database query results, and binary blobs.

Key features include simple LRU expiration, lazy eviction, and client‑side distribution (the server itself has no built‑in clustering).

Architecture diagram:

Redis

Redis is a remote in‑memory data store (NoSQL) that supports multiple data types, persistence, replication, sharding, and high‑availability features such as Sentinel and Cluster.

It provides rich data structures, Lua scripting, transaction support, and configurable eviction policies.

Data model illustration:

Eviction strategy diagram:

Internal hash‑table implementation (gradual rehash) and other low‑level details are shown in subsequent diagrams.

Cache design principles and a direct comparison with Memcached are illustrated:

Cache Architecture Design Challenges

The article enumerates common problems in layered cache architectures and proposes solutions.

Data Consistency – Inconsistent cache copies can cause stale reads; solutions include read‑through/write‑through patterns and versioning.

Cache Penetration – Repeated misses for non‑existent keys overload the database; mitigate by caching empty results and using Bloom filters.

Cache Avalanche – Simultaneous expiration of many keys overwhelms the backend; staggered TTLs and pre‑warming are typical mitigations.

High Availability – Design must match business criticality; techniques include data distribution and replication across nodes.

Cache Hotspots – Hot data accessed concurrently can saturate a single cache node; replicate the hotspot across multiple nodes and employ load‑balancing.

Data Consistency

Because caches are replicas of persistent data, network partitions or node failures can lead to stale or missing data. The article suggests using consistent hashing, version stamps, and fallback reads from the source database.

Cache Penetration

When a key never exists, repeated database queries occur. Solutions: cache null results with a short TTL and employ a Bloom filter to filter out guaranteed‑absent keys.

Cache Avalanche

Mass expiration can cause a sudden surge of database traffic. Recommended tactics include randomizing TTLs, using warm‑up scripts, and employing multi‑level caches.

High Availability

Depending on the service’s tolerance, designers can choose distributed deployment and replication to ensure failover capability.

Cache Hotspots

Hot data should be replicated across several cache nodes, and request routing should spread the load evenly.

Industry Case Study

The article references a Sina Weibo technical presentation that combines Redis/Memcached with an SSD‑based L2 cache in front of MySQL. This architecture reduces cost, alleviates DB pressure caused by cache penetration, and improves overall performance and scalability.

Additional diagrams illustrate the feed cache architecture and its advantages.

Overall, the article serves as a detailed guide for engineers designing, implementing, and operating cache layers in high‑traffic distributed systems.