Choosing the Right Cache: Local vs Distributed Strategies Explained

1 Local Cache JDK Map

JDK Map implementations are often used for cache, such as HashMap, ConcurrentHashMap, LinkedHashMap, and TreeMap.

HashMap provides fast insert, lookup, and delete operations, using key‑value pairs as cache entries.

ConcurrentHashMap is thread‑safe and supports efficient concurrent reads and writes.

LinkedHashMap maintains insertion order and can be traversed by insertion or access order.

TreeMap is an ordered map based on a red‑black tree, allowing traversal by key order.

In a red‑packet system, activities were stored in a ConcurrentHashMap and refreshed by a scheduled task.

Core process: initialize a ConcurrentHashMap, load all activities from the database into the map, and refresh the cache every 30 seconds.

Advantages of this approach include fast response for high‑concurrency scenarios, limited memory usage due to few activities, and non‑impactful cache refresh.

2 Local Cache Frameworks

While JDK Map is quick to build a cache, its functionality is limited. Real‑world scenarios often require cache statistics, expiration, and eviction policies, leading to the emergence of local cache frameworks.

Popular Java cache frameworks include Ehcache, Google Guava, and Caffeine.

Caffeine usage example:

However, local caches have clear drawbacks: after a restart the cache is empty, causing high load while rebuilding, and each node maintains its own cache, making synchronization difficult.

3 Distributed Cache

Distributed cache spreads data across multiple machines, increasing capacity and concurrent read/write capability. Redis is the de‑facto choice for distributed caching.

Compared with local cache, distributed cache offers scalability and high availability, but introduces network latency and added complexity such as serialization, sharding, and cache sizing.

A real case: a live‑score system using Memcached suffered frequent GC pauses due to large cache objects (300‑500 KB). The solution was to increase the young generation heap size and split the cache into full data and incremental data, delivering updates via WebSocket.

4 Multi‑Level Cache

Open‑Source China initially used Ehcache, but frequent restarts cleared the cache and overloaded the database. They built a multi‑level cache (J2Cache) combining Ehcache (local) and Redis (distributed).

Benefits of multi‑level caching:

Closer to the user, faster response.

Reduces distributed cache queries, lowering CPU cost of serialization.

Significantly cuts network I/O and bandwidth usage.

In 2018, an e‑commerce app’s homepage was optimized with a two‑level cache using Guava’s lazy loading. The flow:

On startup, the local cache is empty; the system reads Redis, and if empty, fetches data via RPC, then writes to both caches.

Subsequent requests read directly from the local cache.

Guava’s refresh thread pool periodically updates both caches from the service.

Problems encountered included data inconsistency across servers due to lazy loading and an undersized thread pool causing task backlog. Solutions were to combine lazy loading with a message‑driven update mechanism and to enlarge and monitor the thread pool.

5 Summary

There is no universal “silver bullet” in software engineering; caching is a double‑edged sword that can boost performance but also adds complexity such as invalidation, updates, and consistency.

When choosing a cache solution, consider business scenarios, development efficiency, operational cost, team expertise, and technical constraints to make a balanced decision.