Mastering Cache Design: Reduce Database Load and Boost High‑Concurrency Performance

Why Design Cache?

High‑concurrency systems face a speed gap between CPU processing and memory access, similar to the CPU cache concept described in computer architecture textbooks. Introducing a fast, small cache between the processor (or application) and memory reduces latency and prevents the memory (or database) from becoming a bottleneck.

As system complexity grows, the performance gap widens, leading to multi‑level caches (L1, L2, L3) in hardware and analogous multi‑tier caches (local JVM cache, Redis cluster) in software.

Reducing Database Load

Relational databases are easy to use but become inefficient as data volume increases. To meet low‑latency, high‑throughput requirements, we must offload frequent read/write operations to caches, thereby decreasing database pressure.

Distributed Multi‑Level Cache

Implement a hierarchy: local in‑process cache for ultra‑fast access, followed by a centralized cache such as Redis for shared data. Follow these principles:

Avoid copy‑paste code; keep cache logic reusable.

Do not tightly couple cache with business logic to ease future maintenance.

During early rollout, use feature switches to diagnose issues, but limit their number and manage them with a unified configuration system (e.g., JD Logistics' UCC).

Spring Cache can provide a simple annotation‑based solution for many of these needs.

Write‑Cache Failure and Consistency

When a cache write fails, maintain eventual consistency by using asynchronous compensation messages. In read‑heavy scenarios, write to the database first, then update the cache.

On database exceptions, rethrow specific exceptions instead of catching generic RuntimeException, enabling targeted error handling.

Performance Considerations

Cache should occupy minimal memory; avoid storing whole objects unnecessarily. Serialization and deserialization add overhead and should be minimized.

Cache Synchronization Strategies

Three common approaches balance consistency, database load, and real‑time requirements:

Lazy Loading

Load data into the cache on read; if the key expires, the next read triggers a database fetch.

Pros: Simple and direct.

Cons: May cause cache miss storms under heavy concurrency.

Refresh (Push) Loading

Write expiration metadata to an async queue; background workers refresh the cache before expiry, often using binlog‑derived incremental updates.

Pros: Smooths traffic spikes by offloading database reads to asynchronous tasks.

Cons: Queue buildup can delay synchronization and increase system complexity.

Timed Loading

Periodically push database snapshots into a centralized cache (e.g., Redis) via scheduled jobs.

Pros: Guarantees low‑latency cache freshness.

Cons: Requires a task scheduler and careful ordering; local caches need additional threads or MQ integration, raising maintenance cost.

Monitor queue sizes and process completed items promptly to avoid scheduling bottlenecks.

Preventing Cache Penetration

Cache penetration occurs when non‑existent keys generate repeated database hits under high concurrency. Mitigation methods include:

Maintain a Set of all valid keys and check membership before querying.

Cache a placeholder (e.g., "0") for missing keys, ensuring subsequent requests hit the cache.

Assign appropriate TTLs to placeholder entries based on business needs.

Hotspot Cache and Eviction Policies

When only a subset of data (e.g., hot‑selling items) needs caching, choose an eviction strategy:

FIFO : Remove the oldest entries first.

LRU : Remove least recently used items; O(1) operations using a hashmap + doubly‑linked list.

LFU : Remove items with the lowest access frequency; may retain stale hot keys if access patterns change abruptly.

Each policy has trade‑offs; LFU is often combined with other strategies to avoid retaining rarely used data.

Common Cache Issues (Q&A)

Q1: When to use local cache vs. centralized cache?

A1: If the data volume is modest and updates are infrequent, a local cache suffices; otherwise, a cluster cache is preferable.

Q2: How to batch‑update many cache entries?

A2: Read data from the database in batches, write them to the cache, and optionally set versioned keys or delete stale entries.

Q3: How to delete unknown keys periodically?

A3: Store all keys in a Redis Set and delete the set in bulk, avoiding costly keys * scans.

Q4: What to do when a single key’s data set is too large for Redis sharding?

A4: Apply TTLs to allow cache misses, or use versioned keys (e.g., timestamp‑based) to trigger refreshes.