Mastering Backend Caching: Strategies, Types, and Common Pitfalls

Cache (Cache) is a storage technique that keeps data for fast retrieval, embodying the principle of "using space to save time" to improve access speed.

Cache Usage in Backend

Improve interface response speed: Cache is much faster than I/O or database queries, so appropriate caching can accelerate responses.

Reduce load on data sources: Frequently read data is cached, allowing repeated requests to be served from cache instead of the database or upstream services.

Save computational resources: Expensive computation results are cached to avoid redundant processing.

Cache Strategies

1.1 Read‑Through Cache

When an application requests data, the cache is checked first. If the data exists, it is returned directly. If not, the cache fetches the data from the source, returns it to the application, and stores it for future requests.

Typical use case: CDN (Content Delivery Network) caches static resources on distributed edge servers to serve users from the nearest node, improving access speed.

1.2 Cache‑Aside (Cache‑as‑Side) Strategy

The application first checks the cache; if the data is missing, it returns empty, then the application fetches the data from the source and writes it into the cache for subsequent requests.

Read‑through and cache‑aside are often combined; a read‑through implementation may internally use cache‑aside for reads and writes.

Cache Types

2.1 Local Cache

Local cache lives in the same process as the application, stored in heap memory.

Advantages: Simple to use, no external dependencies, fast reads (no network I/O).

Disadvantages: Limited space, possible inconsistency in distributed systems, cannot persist after process exit.

2.2 Distributed Cache (Remote Cache)

Distributed cache is an external service deployed independently, decoupled from the application.

Advantages: Sufficient storage, no consistency issues across instances, mainstream solutions like Redis support persistence and high availability.

Disadvantages: Introduces external dependency and requires deployment and operation.

2.2.1 Common Cache Services

Redis

Supports multiple data structures (list, set, zset, hash, etc.).

Provides data persistence and recovery.

Offers clustering, master‑slave replication, and Sentinel for high availability.

Delivers millisecond‑level response times.

Redis 6.0 introduced multithreaded I/O.

Memcached

Simple key‑value storage.

No built‑in persistence.

Supports sharding for distributed storage.

Uses multithreaded, non‑blocking I/O.

Provides millisecond‑level response times.

Choosing between Redis and Memcached depends on application needs: Redis for rich data types and persistence, Memcached for lightweight caching.

Cache Eviction Policies

3.1 FIFO (First In First Out)

FIFO evicts the oldest inserted entries first, assuming older data is less likely to be accessed.

3.2 LRU (Least Recently Used)

LRU removes the data that has not been accessed for the longest time, typically implemented with a doubly linked list and a hash map.

3.3 LFU (Least Frequently Used)

LFU evicts the data with the lowest access frequency, often implemented with a min‑heap and a hash map.

Most third‑party cache libraries allow configuring these eviction algorithms without custom implementation.

Common Cache Problems

4.1 Consistency with Data Source

When the data source updates but the cache does not, stale reads occur. Two main strategies address this:

Expiration (TTL): Set an appropriate TTL; after expiration the cache fetches fresh data. Suitable for infrequently changing data where slight delay is acceptable.

Proactive Update: Update the cache immediately when the source changes. Needed for high‑frequency data like inventory or balances, but increases code complexity.

4.2 Cache Avalanche

A large number of keys expire simultaneously, causing a sudden surge of traffic to the data source, potentially overwhelming it.

Mass simultaneous expiration of many keys.

Cache service outage forcing all requests to hit the database.

Set staggered TTLs to avoid simultaneous expiration.

Ensure high availability of the cache service (e.g., clustering, failover).

4.3 Cache Penetration

Massive requests for non‑existent keys bypass the cache because empty results are not cached, leading to repeated database hits.

Insufficient request validation.

Deliberate malicious attacks.

Validate request parameters and filter illegal inputs.

Cache empty results with a short TTL or a sentinel value.

4.4 Cache Stampede (Cache Breakdown)

When a hot key expires, a flood of concurrent requests hits the data source, potentially causing overload.

Hot data expires while many threads request it simultaneously.

Use mutex or distributed locks to ensure only one request fetches from the source.

Keep hot data in cache permanently with proactive updates.

Pre‑load hot data before expiration.

Conclusion

This article thoroughly introduces caching techniques for backend development, emphasizing the importance of mastering various strategies and mechanisms to enhance efficiency, stability, and user experience.