How Guava Cache Works Under the Hood: Expiration, LRU, and Implementation Details

Overview

Guava Cache is a Java in‑memory caching library that adds automatic eviction, configurable expiration policies, and removal callbacks on top of the standard ConcurrentHashMap. It is built using a builder pattern, allowing developers to fine‑tune concurrency level, maximum size, time‑based expiration, and reference handling.

Key Features

Automatic eviction : entries can be removed based on size (LRU, LFU, FIFO) or time ( expireAfterWrite, expireAfterAccess).

Removal listeners : a user‑supplied callback is invoked when an entry is evicted or manually removed.

Concurrency : the cache uses a segmented ConcurrentHashMap with fine‑grained locks, and a volatile counter to track the total number of entries safely across threads.

Typical Use‑Case: Rate‑Limiting Alerts from Kafka

Read real‑time logs from Kafka; if X exceptions occur within a time window N, trigger an alert.

The cache stores a counter per key and automatically expires the counter after N minutes of inactivity, simplifying the implementation of sliding‑window rate limiting.

@Value("${alert.in.time:2}")
private int time;

@Bean
public LoadingCache<Long, AtomicLong> buildCache() {
    return CacheBuilder.newBuilder()
        .expireAfterWrite(time, TimeUnit.MINUTES)
        .build(new CacheLoader<Long, AtomicLong>() {
            @Override
            public AtomicLong load(Long key) {
                return new AtomicLong(0);
            }
        });
}

public void checkAlert() {
    try {
        if (counter.get(KEY).incrementAndGet() >= limit) {
            LOGGER.info("*********** ALERT ***********");
            counter.get(KEY).set(0L); // reset after alert
        }
    } catch (ExecutionException e) {
        LOGGER.error("Exception", e);
    }
}

Each log consumption calls checkAlert(). The expireAfterWrite policy clears the counter if no new logs are processed within the configured minutes, preventing stale data from accumulating.

Internal Implementation Details

Guava’s cache implementation resides in com.google.common.cache.LocalCache. The core eviction flow works as follows:

When a cache operation (e.g., get) occurs, the code checks the volatile int count field (line ~2182) to ensure the cache is not empty.

It then evaluates whether the specific entry is expired (line ~2761) based on the expiration policy supplied during construction.

If the entry is expired, Guava acquires the segment lock, decrements count atomically, removes the entry from the underlying map, and updates the counter.

There is no dedicated cleanup thread; expiration is performed lazily during normal cache accesses. This design avoids extra thread overhead and lock contention, at the cost that entries never accessed after expiration may remain in memory until the next operation touches the segment.

Data Structures

Segments : the cache is divided into several segments (similar to ConcurrentHashMap) to reduce lock contention. A key is first hashed to locate the segment, then hashed again to find the bucket within that segment.

Access and Write Queues : two linked queues ( accessQueue and writeQueue) maintain entry order for LRU‑style eviction, analogous to the ordering in LinkedHashMap.

Concurrency Controls

volatile

fields (e.g., count) guarantee visibility of updates across threads.

Fine‑grained segment locks protect modifications to individual buckets without blocking the entire cache.

Builder Pattern

Cache instances are created via CacheBuilder, which offers methods such as: maximumSize(long) – size‑based eviction using LRU. expireAfterWrite(long, TimeUnit) – time‑based eviction after the last write. expireAfterAccess(long, TimeUnit) – time‑based eviction after the last read or write. removalListener(RemovalListener) – register a callback for evicted entries.

These options are combined fluently before calling build() or build(CacheLoader) to obtain a Cache or LoadingCache instance.

Summary

Guava Cache provides a high‑performance, thread‑safe in‑memory cache built on top of ConcurrentHashMap. It achieves automatic expiration without a background thread by performing cleanup during normal cache operations, using volatile counters and segment‑level locking to maintain consistency. The builder API makes it easy to configure size limits, time‑based eviction, and removal listeners, while the internal access/write queues enable LRU‑style eviction.