Mastering Cache Strategies: From Local to Multi‑Level for High‑Performance Systems

01 Local Cache

Early in my career I used OSCache for page‑level caching in JSP. The cache tag looks like:

<cache:cache key="foobar" scope="session">
    some jsp content
</cache:cache>

The snippet stores the rendered fragment under foobar in the session, allowing other pages to reuse it. With the rise of front‑end separation, page‑level server caching fell out of favor, but it remains popular on the front‑end side.

Object Cache

Inspired by a 2011 article on Open‑Source China, I introduced Ehcache into a balance‑withdrawal service to cache the success/failure state of orders, avoiding repeated calls to Alipay. The pseudo‑code above shows how the cache key is stored in the session.

After adding the cache, processing time dropped from 40 minutes to 5‑10 minutes.

Refresh Strategies

In 2018 we built a configuration center that uses Guava for local caching. The architecture is illustrated below:

Two mechanisms keep the local cache fresh:

Clients start a scheduled task that pulls data from the configuration center.

When the configuration changes, the center pushes updates via RocketMQ Remoting.

02 Distributed Cache

Memcached and Redis are the most common distributed caches. Two real‑world cases are presented.

Control Object Size & Read Strategy

In a lottery‑score streaming service (2013) large cache values (300‑500 KB) caused frequent Young GC pauses. Reducing the cached object size and adjusting the new‑generation heap from 2 GB to 4 GB mitigated the issue, but large objects still stressed the system.

Pagination List Cache

Two approaches are described for caching paginated blog lists.

Cache the whole page: the cache key combines page number and size; the value is the full list. Updating a single blog forces a full‑page invalidation.

Cache individual blog entries: first query the IDs for the requested page, then batch‑fetch the corresponding entries from the cache. Missed IDs are retrieved from the database and written back to the cache.

SQL examples:

select id from blogs limit 0,10;

select id from blogs where id in (noHitId1, noHitId2);

Batch fetching can use mget for Memcached or mget/hmget or pipelines for Redis.

03 Multi‑Level Cache

Multi‑level caching combines fast, limited‑capacity local caches with scalable distributed caches. The principle is “the closer the cache to the user, the more efficient it is.”

In a 2018 e‑commerce app we applied a two‑level cache: Guava local cache with lazy loading and Redis as the remote layer. The flow:

On service start, the gateway reads from Redis; if empty, it fetches data via RPC, then populates both Redis and the local cache.

Subsequent requests read directly from the local cache.

Guava’s refresh thread (5 core, 5 max threads) periodically pulls fresh data from the upstream service and updates both caches.

After deployment the average latency was ~5 ms, but occasional inconsistencies appeared because lazy loading caused divergent data across instances and the LoadingCache thread pool was undersized.

Lessons Learned & Fixes

Lazy loading can still produce inconsistent data across machines.

Thread‑pool size for cache refresh must be tuned and monitored.

Combine lazy loading with a push‑based notification (e.g., via message bus) so that configuration changes trigger immediate cache invalidation.

Conclusion

Caching is a vital technique for performance. Understanding its principles—from local page caches to distributed and multi‑level designs—helps engineers avoid common pitfalls such as oversized objects, GC pressure, and data inconsistency. Future posts will explore high‑availability cache patterns and the internals of Codis.