Mastering Cache: Core Concepts, Pitfalls, and Patterns for Scalable Systems

1. Basic Idea of Caching

Many people know that caching can improve system performance and reduce response time, but they often do not understand the essential concept.

The core idea is simple: it is the classic space‑for‑time trade‑off. Caching offers a high cost‑performance ratio for speeding up data access.

The same principle appears throughout computing, e.g., CPU cache stores memory data to bridge the speed gap between CPU and RAM, and disk cache stores data to mitigate slow disk access. Operating systems also use a fast lookup table (TLB) as a special cache.

In business systems we add a cache layer above the database to avoid slow data‑fetch operations for users.

2. Problems Introduced by Caching

There is no silver bullet in software design; any technology can be a double‑edged sword. Proper use yields great benefits, misuse wastes effort.

Introducing cache can bring several issues, especially with distributed caches:

Increased system complexity : you must maintain data consistency between cache and database and handle hot keys.

Higher development cost : a separate cache service consumes expensive memory and operational resources.

3. Local Cache Solutions

Local cache is common in monolithic architectures when data volume is small and no distributed requirements exist.

Typical setup: two identical services behind Nginx load balancer share one database and each uses its own local cache.

Common local‑cache options:

JDK built‑in : HashMap and ConcurrentHashMap. They store key/value pairs but lack expiration, eviction, and hit‑rate statistics.

Third‑party libraries : Ehcache, Guava Cache, Spring Cache. Ehcache is heavier but supports integration with Hibernate/MyBatis, persistence to disk, and optional clustering. Guava Cache and Spring Cache are similar; Guava offers a convenient API and expiration features, while Spring Cache provides annotation‑based configuration but can suffer from cache‑penetration and memory overflow.

Caffeine : a newer library that outperforms Guava in speed and offers a similar usage model.

Local cache is lightweight, low‑cost, and simple, but it cannot be shared across multiple service instances.

4. Why Use Distributed Cache?

Distributed cache (e.g., Redis, Memcached) offers low dependency, light weight, simplicity, and low cost, yet it solves two major drawbacks of local cache:

It does not support distributed architectures; each instance’s cache is isolated.

Cache capacity is limited by the memory of a single host.

Distributed cache acts as an in‑memory database service that can be shared by all service instances.

Deploying a separate cache server allows many instances to use the same cache, providing higher performance, larger capacity, and richer features, at the cost of managing an additional high‑availability service.

5. Cache Read/Write Patterns and Update Strategies

Three common patterns each have trade‑offs; choose based on business needs.

5.1 Cache‑Aside (Lazy Loading)

Write :

Update the database first.

Delete the corresponding cache entry.

Read :

Try to read from cache; if hit, return.

If miss, read from database, then populate cache.

Potential issues: first request always misses; frequent writes cause cache invalidations and lower hit rates. Mitigations include pre‑warming hot data and using locks or short TTLs for consistency.

5.2 Read/Write‑Through

The cache is treated as the primary store; it reads from and writes to the database on behalf of the application.

Write :

If key absent in cache, write directly to DB.

If present, update cache first, then the cache service synchronously updates the DB.

Read :

Return data from cache if present.

Otherwise load from DB, write into cache, and return.

This pattern is essentially a wrapper around Cache‑Aside that makes the cache service handle the write‑back automatically.

5.3 Write‑Behind (Asynchronous Write)

Similar to Write‑Through but updates the database asynchronously after writing to cache.

Advantages: very high write throughput, suitable for scenarios where strict consistency is not required (e.g., view counts, likes). Drawbacks: risk of data loss if the cache crashes before flushing to DB.

Use cases include message‑queue persistence and MySQL InnoDB buffer‑pool mechanisms.

Source: https://www.cnblogs.com/xzsj/p/gzxzsj-database-redis.html