Choosing the Right Local Cache: From ConcurrentHashMap to Caffeine

Background

In high‑performance service architectures, hot data is typically stored in remote caches such as Redis or Memcached. When a cache miss occurs the request falls back to the database. To further reduce latency and database load, a two‑level cache adds a local (first‑level) cache in front of the remote (second‑level) cache.

Why Use Local Cache

Local cache resides in process memory, providing ultra‑fast access for data with low change frequency and low real‑time requirements.

It reduces network I/O and latency by avoiding unnecessary round‑trips to remote caches.

Design Requirements for a Local Memory Cache

Store key‑value pairs with read/write capability.

Thread‑safe atomic operations (e.g., using ConcurrentHashMap).

Configurable maximum size.

Eviction policies (LRU, LFU) when the size limit is exceeded.

Expiration strategies (time‑based, lazy, periodic).

Optional persistence support.

Statistics and monitoring.

Local Cache Implementation Options

1. ConcurrentHashMap

Using a plain ConcurrentHashMap is the simplest approach and requires no third‑party dependencies. It is suitable for very basic scenarios, but eviction, size limits and expiration must be implemented manually, which increases development effort and risk.

2. Guava Cache

Guava provides a mature cache implementation with built‑in size limits, expiration policies, basic statistics and LRU‑based eviction.

Maximum size limit.

Expiration after write or after access.

Basic hit/miss statistics.

LRU eviction.

Example (Maven dependency and usage):

<dependency>
  <groupId>com.google.guava</groupId>
  <artifactId>guava</artifactId>
  <version>31.1-jre</version>
</dependency>

@Slf4j
public class GuavaCacheTest {
    public static void main(String[] args) throws ExecutionException {
        Cache<String, String> cache = CacheBuilder.newBuilder()
                .initialCapacity(5)
                .maximumSize(10)
                .expireAfterWrite(60, TimeUnit.SECONDS)
                .build();
        String orderId = String.valueOf(123456789);
        String orderInfo = cache.get(orderId, () -> getInfo(orderId));
        log.info("orderInfo = {}", orderInfo);
    }
    private static String getInfo(String orderId) {
        log.info("get data from redis");
        log.info("get data from mysql");
        return String.format("{orderId=%s}", orderId);
    }
}

3. Caffeine

Caffeine is a Java‑8‑based cache library that uses the W‑TinyLFU algorithm (a hybrid of LRU and LFU) and delivers near‑optimal performance.

Example (Maven dependency and usage):

<dependency>
    <groupId>com.github.ben-manes.caffeine</groupId>
    <artifactId>caffeine</artifactId>
    <version>2.9.3</version>
</dependency>

@Slf4j
public class CaffeineTest {
    public static void main(String[] args) {
        Cache<String, String> cache = Caffeine.newBuilder()
                .initialCapacity(5)
                .maximumSize(10)
                .expireAfterWrite(60, TimeUnit.SECONDS)
                .build();
        String orderId = String.valueOf(123456789);
        String orderInfo = cache.get(orderId, key -> getInfo(key));
        System.out.println(orderInfo);
    }
    private static String getInfo(String orderId) {
        log.info("get data from redis");
        log.info("get data from mysql");
        return String.format("{orderId=%s}", orderId);
    }
}

4. Ehcache (Encache)

Ehcache is a pure‑Java in‑process cache framework that offers multiple storage tiers (heap, off‑heap, disk) and a richer set of eviction algorithms.

Eviction algorithms: LRU, LFU, FIFO.

Three storage tiers: heap, off‑heap, and disk (with persistence).

Clustering options for data sharing.

Example (Maven dependency and usage):

<dependency>
   <groupId>org.ehcache</groupId>
   <artifactId>ehcache</artifactId>
   <version>3.9.7</version>
</dependency>

@Slf4j
public class EhcacheTest {
    private static final String ORDER_CACHE = "orderCache";
    public static void main(String[] args) {
        CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder()
                .withCache(ORDER_CACHE, CacheConfigurationBuilder
                        .newCacheConfigurationBuilder(String.class, String.class, ResourcePoolsBuilder.heap(20)))
                .build(true);
        Cache<String, String> cache = cacheManager.getCache(ORDER_CACHE, String.class, String.class);
        String orderId = String.valueOf(123456789);
        String orderInfo = cache.get(orderId);
        if (orderInfo == null) {
            orderInfo = getInfo(orderId);
            cache.put(orderId, orderInfo);
        }
        log.info("orderInfo = {}", orderInfo);
    }
    private static String getInfo(String orderId) {
        log.info("get data from redis");
        log.info("get data from mysql");
        return String.format("{orderId=%s}", orderId);
    }
}

Cache Consistency Strategies

MQ Broadcast

In a clustered deployment, each node subscribes to a message‑queue broadcast. When data changes, a message is sent to all nodes to evict the corresponding local cache entry, achieving eventual consistency.

Canal + MQ

If embedding MQ logic in business code is undesirable, the database binlog can be captured (e.g., with Canal for MySQL). Changes are published to MQ, and downstream services invalidate the local cache accordingly.

Improving Local Cache Hit Rate

Techniques such as thoughtful key design, cache warming, and read‑through/write‑through patterns can increase the hit rate.

Technical Selection Considerations

Ease of use: Guava Cache, Caffeine and Ehcache all provide mature APIs.

Feature set: Guava and Caffeine support heap‑only caching; Ehcache adds off‑heap, disk persistence and clustering.

Performance: Benchmarks show Caffeine > Guava Cache > Ehcache.

Recommendation

Use Caffeine as the local cache due to its superior performance, and combine it with Redis or Memcached as a distributed second‑level cache to build a robust multi‑level caching system.