Mastering Cache Strategies: From Browser to Distributed Systems

Browser Cache

Browsers cache non‑time‑critical data such as page frameworks, merchant ratings, reviews, and ad copy to speed up surfing. The cache respects expiration settings controlled via the HTTP Expires and Cache‑Control response headers.

Client Cache

Client‑side cache stores resources on the device before a traffic surge, such as during a promotion. By pre‑loading JavaScript, CSS, images, and fallback data, the app avoids fetching them from the server during peak load, improving resilience against server or network failures.

CDN Cache

A Content Delivery Network (CDN) consists of edge nodes distributed across regions. Static pages, promotional pages, and images are stored in CDN caches. CDN caching works via two mechanisms: push (proactively uploading updated content) and pull (fetching from the origin when a request misses).

Analogy: a car dealership acts like a CDN node—if the car is on the lot, the customer gets it immediately; otherwise the dealership must order it from the manufacturer.

Reverse‑Proxy Cache (Nginx)

Nginx provides two cache layers: HTTP cache and proxy‑layer cache. The HTTP cache uses directives such as expires, etag, and if‑modified‑since. The proxy cache is configured via the proxy_cache module.

Local Cache

Local cache refers to memory allocated on a machine to buffer data before writing it to persistent storage. It includes:

Disk cache (read and write caches)

CPU cache (L1, L2, L3)

Application cache (in‑process caches)

Disk Cache

Read cache keeps recently accessed file data in a memory pool, allowing faster subsequent reads. Write cache buffers data before flushing it to disk when the buffer reaches a threshold.

CPU Cache

Data lookup follows the hierarchy: CPU → L1 → L2/L3 → main memory → disk.

Application Cache

Application‑level caches can be in‑heap or off‑heap. In‑heap caches (e.g., Guava Cache, Ehcache) store objects directly in the Java heap, offering fast access but increasing GC pressure and losing data on restart. Off‑heap caches reside outside the heap, avoiding GC impact but requiring serialization, which adds latency and risk of memory leaks.

Other application caches include local Redis instances deployed on the same server, providing ultra‑low latency without network overhead.

Distributed Cache

When a single machine cannot meet capacity or consistency requirements, data is sharded across multiple nodes to form a distributed cache.

Key challenges of a local Redis cache:

Limited capacity per machine

Data consistency across multiple instances

Reduced cache‑hit rate leading to increased DB load

Typical sharding strategies include:

Modulo of node identifier

Consistent hashing

Virtual slot partitioning (used by Redis Cluster)

Cache Hit Rate

The cache hit rate measures how often requested data is served from the cache rather than the backing store. A higher hit rate indicates better cache effectiveness.

Improvement tactics include:

Pre‑warming hot data before traffic spikes

Separating hot and cold data, with real‑time hot‑data detection

Assigning appropriate TTLs per data category

Reducing cache granularity to increase specificity

Expanding cache capacity to avoid premature evictions

Cache Problems

Cache Breakdown (Cache Penetration)

When both the cache and the database lack a requested key, a flood of requests can hit the database directly, risking overload.

Cache Penetration

Occurs when a key expires in the cache but still exists in the database; massive concurrent requests for that key bypass the cache and stress the DB.

Cache Avalanche

Triggered when many keys expire simultaneously, causing a sudden surge of DB queries that can overwhelm the database.

Cache Consistency

Ensures that cached data remains synchronized with the underlying database, often requiring write‑through, write‑behind, or explicit invalidation strategies to achieve eventual or strong consistency.

Other Cache Challenges

Additional issues include cache skew, blocking, slow queries, master‑slave consistency, high availability, fault detection and recovery, cluster scaling, and handling large or hot keys.