Effective Cache Strategies for Large Distributed Systems

In large distributed systems, proper cache design is essential to avoid catastrophic load on the data layer; the article shares practical experiences from a high‑traffic live‑streaming product and discusses how different cache layers can be used efficiently.

Client Cache

Client‑side caching, typically implemented via HTTP headers, reduces network latency and request volume. The article shows a Spring ResponseEntity example:

ResponseEntity.ok().cacheControl(CacheControl.maxAge(3, TimeUnit.SECONDS)).body()

and the required imports:

import org.springframework.http.CacheControl;
import org.springframework.http.ResponseEntity;

Key HTTP directives include Cache‑Control: max‑age , no‑store , no‑cache , and must‑revalidate . Conditional requests using Last‑Modified and ETag allow validation with 304 Not Modified responses.

Server Cache

CDN Cache

CDN caches static assets (images, audio, video) and can also cache infrequently changing dynamic data. Configuration is usually done at the service‑governance layer without code changes.

Redis Cache

Redis is the most common distributed cache. The article reviews its five core data types (String, List, Set, Sorted Set, Hash) and shows how they solve typical use cases such as leaderboards, set operations, and bitmap‑based counters.

Example of a leaderboard using a Sorted Set is described, as well as set difference for daily new‑user calculation using SDIFFSTORE . Bitmap commands SETBIT , GETBIT , and BITCOUNT are highlighted for space‑efficient counting.

Common Redis challenges include data consistency, the three cache problems (penetration, breakdown, avalanche), network timeouts, and high‑complexity operations on large keys.

Data Consistency

Read‑only caches risk stale data; two update patterns are discussed (update DB then delete cache, or delete cache then update DB) with their pitfalls under high concurrency. The recommended approach is to update the database first and then delete the cache, optionally using retry mechanisms or short TTLs.

Cache Penetration, Breakdown, Avalanche

Penetration can be mitigated by caching null objects or using Bloom filters. Breakdown is addressed by locking the hot key during cache miss or avoiding TTL for hot keys. Avalanche is prevented by high‑availability clusters and staggered expiration times.

Interface Timeout & High Complexity

Network latency can cause Redis timeouts; retry logic and appropriate timeout settings are advised. Operations with O(N) complexity on large collections may cause “big‑key” issues; splitting keys or sharding (e.g., CRC16 modulo 16384) can alleviate the problem.

Cache Update via Message Queues

Asynchronous cache updates can be performed by publishing change events to a message queue; consumers update the cache after processing, ensuring eventual consistency.

Scheduled Tasks

Local caches refreshed by scheduled jobs reduce load on remote services. In Spring Boot, @EnableScheduling and @Scheduled(cron = "0 0 2 * * ?") are used to configure periodic refreshes.

Local Cache with @Cacheable

Spring's @Cacheable and @CachePut annotations provide declarative caching. For more control, Guava's LoadingCache can be configured with size limits and expiration policies, as shown in the code snippet:

private final LoadingCache<String, List<ParentTag>> tagCache = CacheBuilder.newBuilder()
    .concurrencyLevel(4)
    .expireAfterWrite(5, TimeUnit.SECONDS)
    .initialCapacity(8)
    .maximumSize(10000)
    .build(new CacheLoader<String, List<ParentTag>>() {
        @Override
        public List<ParentTag> load(String cacheKey) {
            return get(cacheKey);
        }
        @Override
        public ListenableFuture<List<ParentTag>> reload(String key, List<ParentTag> oldValue) {
            ListenableFutureTask<List<ParentTag>> task = ListenableFutureTask.create(() -> get(key));
            executorService.execute(task);
            return task;
        }
    });

The article concludes that a combination of client, CDN, and server caches—especially Redis for shared data and local caches for fast access—should be chosen based on cost, consistency requirements, and performance characteristics.