Why Every Project Needs Caching – Master Local and Distributed Cache Basics

Why caching is essential

Memory reads are measured in nanoseconds while a database query can take tens of milliseconds, a speed gap of over ten thousand times. The primary purpose of caching is to reduce database queries, handle high concurrency, and achieve large performance gains.

Core principle of caching

Cache = store high‑frequency data in a faster‑to‑read location. By placing frequently accessed data in memory or a fast‑access service, applications avoid the costly latency of disk or database access.

Two major cache categories

1. Local cache (in‑process memory)

Definition: Deployed on the current service instance and isolated to a single application process.

Common implementations: Map, Guava Cache, Caffeine.

Advantages:

Zero network request, fastest speed, zero latency.

No third‑party component, simple, stable, no extra overhead.

Disadvantages:

Data not shared across instances, leading to inconsistency.

Consumes local memory; large data sets can cause OOM.

Suitable scenarios: Low‑frequency changes, static dictionaries, configuration, constants.

2. Distributed cache (standalone middleware)

Definition: Independently deployed cache service shared by all application instances.

Common implementations: Redis, Memcached (Redis is the mainstream choice).

Advantages:

Global data sharing, unified across services.

Separate deployment, does not consume business service memory.

Supports massive data, expiration policies, persistence, and high‑concurrency adaptation.

Disadvantages:

Requires network transmission, slightly slower than local cache.

Depends on middleware, adds operational cost.

Suitable scenarios: High‑frequency hot data, user sessions, flash‑sale inventory, frequently queried interface data.

Best practice: Two‑level cache architecture

Real‑world projects combine local cache and distributed cache.

Query flow:

Check local cache first; if hit, return immediately.

If local miss, query Redis distributed cache.

If Redis miss, fall back to the database.

After a successful query, write‑back to both caches for the next request.

Benefit: Balances extreme speed, data consistency, and high‑concurrency handling.

Interview‑focused cache pitfalls

1. Cache penetration

Phenomenon: Queries for non‑existent data never hit the cache, causing every request to hit the database.

Example: Maliciously sending a large number of nonexistent IDs overwhelms the DB.

Solution: Cache null values, use Bloom filters, or perform parameter validation.

2. Cache breakdown

Phenomenon: When a hot key expires, a surge of concurrent requests all hit the database.

Example: Home‑page hot data expires, traffic instantly floods the DB.

Solution: Use mutex locks, keep hot keys from expiring, or pre‑warm with scheduled refresh.

3. Cache avalanche

Phenomenon: Massive numbers of keys expire simultaneously, causing a wave of DB requests that can crash the service.

Solution: Add random jitter to expiration times, employ multi‑level caching, or apply request rate limiting.