In‑Memory Cache Design with Guava LoadingCache, Eviction Strategies, and LRU Implementation

In the previous article "浅谈缓冲区" the concept of buffering was introduced; this article focuses on its sibling, caching, which is one of the most widely used optimization techniques in software, from CPU multi‑level caches to Redis‑style distributed caches.

Caching improves performance by placing frequently accessed data in a faster storage tier, reducing page load times and relieving pressure on downstream databases.

In typical applications caches are classified as in‑process (heap) or out‑of‑process. This article concentrates on in‑process (heap) caches in Java, such as Ehcache, JCache, Caffeine, and Guava.

Guava LoadingCache is a convenient heap cache. It can be added via Maven:

<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>31.1-jre</version>
</dependency>

Typical configuration parameters include maximumSize (capacity limit), initialCapacity , and concurrencyLevel . Setting the size too small causes frequent evictions; setting it too large wastes memory and adds GC pressure.

Common cache operations are:

Manual put of an object.

Lazy loading via a CacheLoader (the "Loading" part of LoadingCache).

Example of a LoadingCache with lazy loading:

public static void main(String[] args) {
    LoadingCache
lc = CacheBuilder
            .newBuilder()
            .build(new CacheLoader
() {
                @Override
                public String load(String key) throws Exception {
                    return slowMethod(key);
                }
            });
}

static String slowMethod(String key) throws Exception {
    Thread.sleep(1000);
    return key + ".result";
}

Cache entries can be removed manually with .invalidate(key) and removal listeners can be attached:

.removalListener(notification -> System.out.println(notification))

Three eviction strategies are covered:

Capacity‑based eviction (LRU) when the cache reaches its maximum size.

Time‑based eviction using expireAfterWrite or expireAfterAccess .

JVM‑GC‑based eviction via weak/soft references (e.g., weakKeys() , weakValues() ).

Memory‑related pitfalls are discussed, such as oversized cache entries causing excessive heap usage and frequent GC cycles.

Cache algorithms:

FIFO – first‑in‑first‑out, simple but rarely optimal.

LRU – least‑recently‑used, the most common choice.

LFU – least‑frequently‑used, combines usage count with recency.

Implementing a basic LRU with LinkedHashMap :

public class LRU extends LinkedHashMap {
    int capacity;
    public LRU(int capacity) {
        super(16, 0.75f, true);
        this.capacity = capacity;
    }
    @Override
    protected boolean removeEldestEntry(Map.Entry eldest) {
        return size() > capacity;
    }
}

Beyond application‑level caches, the OS page cache (the "cached" column in free ) stores file data in memory, and the kernel uses readahead to pre‑fetch sequential blocks, further speeding up I/O.

When to adopt a cache:

Data exhibits hot spots and is read far more often than written.

Downstream services are slower or have limited capacity.

Introducing the cache does not compromise correctness or add unmanageable complexity.

Key metrics: a hit rate above 50 % yields noticeable benefits; below 10 % suggests the cache may be unnecessary.

In summary, the article covered Guava LoadingCache usage, common eviction policies, memory pitfalls, cache algorithms (FIFO, LRU, LFU), a simple LRU implementation, OS‑level caching techniques, and practical guidelines for deciding when to employ caching in backend systems.