Mastering Cache: From Local to Multi‑Level Strategies for High‑Performance Systems

1. Local Cache

1.1 Page‑level Cache

The author began using OSCache around 2010 to cache JSP page fragments, storing content in the session with a key such as

<cache:cache key="foobar" scope="session"> ... </cache:cache>

. This dramatically reduced page load time in legacy monolithic applications.

1.2 Object Cache

Inspired by a case where a single server handled millions of requests using Ehcache, the author introduced object caching for order status in a payment‑withdrawal service. By caching success/failure results, query latency dropped from 40 minutes to 5‑10 minutes. Object cache is ideal for rarely changing data like global configs or completed orders.

1.3 Refresh Strategies

In a custom configuration center, Guava was used for local cache with two update mechanisms: (1) a scheduled task pulls updates from the center; (2) the center pushes changes via RocketMQ Remoting. The author also examined Soul gateway’s three strategies—Zookeeper watch, WebSocket push, and HTTP long‑polling—each ensuring local caches stay consistent.

2. Distributed Cache

2.1 Controlling Object Size and Access Patterns

During a lottery live‑score service, large cache values (300‑500 KB) caused frequent Young GC pauses. Reducing the cached JSON payload to a compact array (e.g., [["2399","小牛","湖人","123"]]) cut average size from ~300 KB to ~80 KB, lowering GC frequency and improving response time.

2.2 Pagination List Caching

Two approaches are described: (1) cache entire pages using a composite key of page number and size; (2) cache individual items by ID. The second method queries IDs from the database, fetches cached items in bulk (using mget for Memcached or hmget /pipeline for Redis), stores misses back into the cache, and finally returns the assembled list. This balances cache hit rate and freshness for dynamic content.

3. Multi‑Level Cache

Local caches are ultra‑fast but limited in capacity; distributed caches scale but can become a bandwidth bottleneck under high concurrency. Combining both—using Guava’s lazy‑load with a refresh thread pool (5 core, 5 max threads) that syncs data from a recommendation service into Redis and local memory—provides sub‑5 ms latency for an e‑commerce app’s homepage.

However, inconsistencies appeared because lazy loading alone does not guarantee data uniformity across instances, and an undersized thread pool caused request backlogs. The final solution merged lazy loading with a push‑based update via messaging, and enlarged the thread pool while adding monitoring and alerts for overload.

Conclusion

Caching remains a vital technique for performance optimization. Mastering its principles—from simple page caches to sophisticated multi‑level architectures—helps engineers build resilient, high‑throughput systems while avoiding common pitfalls such as oversized objects, stale data, and thread‑pool misconfiguration.