Designing a Robust Local Cache in Java: Key Considerations and Implementation

Introduction

MyBatis provides first‑level and second‑level caches. The second‑level cache demonstrates a feature‑rich local cache that can be used as a reference for implementing a custom in‑memory cache.

Design Considerations

1. Data structure

Store cached entries in a Map. For thread‑safe access most implementations use java.util.concurrent.ConcurrentHashMap. Example:

Map<Object, Object> cache = new ConcurrentHashMap<>();

2. Maximum entry count

Define a hard limit (e.g., 1024 entries). When the limit is reached the cache must evict entries according to an eviction policy.

3. Eviction policies

LRU – Least Recently Used; typically implemented with LinkedHashMap in access‑order mode.

FIFO – First In First Out; can be realized with a Queue that tracks insertion order.

LFU – Least Frequently Used; requires a secondary map that records access counts.

SOFT – Uses java.lang.ref.SoftReference so the GC may reclaim entries under memory pressure.

WEAK – Uses java.lang.ref.WeakReference for more aggressive reclamation.

4. Expiration (TTL)

Each entry may have a time‑to‑live. Two removal strategies are common:

Passive expiration – get or put checks the timestamp and removes the entry if it is stale.

Active expiration – a background thread periodically scans the map and evicts expired entries.

if (System.currentTimeMillis() - lastClear > clearInterval) {
    clear();
}

5. Thread safety

Use concurrent containers or wrap a plain HashMap with synchronized methods. MyBatis provides SynchronizedCache as an example:

public synchronized void putObject(Object key, Object value) { … }
public synchronized Object getObject(Object key) { … }

6. Minimal public API

A cache should expose a small set of operations:

public interface Cache {
    String getId();
    void putObject(Object key, Object value);
    Object getObject(Object key);
    Object removeObject(Object key);
    void clear();
    int getSize();
    java.util.concurrent.locks.ReadWriteLock getReadWriteLock();
}

Guava’s Cache interface adds bulk operations and statistics but follows the same core idea.

7. Persistence (optional)

Persisting entries to disk enables warm starts after a process restart. For example, Ehcache can write to a .data file by setting diskPersistent="true". Redis offers AOF and RDB persistence.

diskPersistent="true"

8. Blocking cache for expensive loads

To avoid duplicate computation on a cache miss, a blocking cache lets only one thread compute the value while others wait. A classic implementation uses a ConcurrentHashMap<A, Future<V>> and FutureTask:

public class Memoizer<A,V> implements Computable<A,V> {
    private final java.util.Map<A, java.util.concurrent.Future<V>> cache =
        new java.util.concurrent.ConcurrentHashMap<>();
    private final Computable<A,V> c;
    public Memoizer(Computable<A,V> c) { this.c = c; }

    @Override
    public V compute(A arg) throws InterruptedException, java.util.concurrent.ExecutionException {
        while (true) {
            java.util.concurrent.Future<V> f = cache.get(arg);
            if (f == null) {
                java.util.concurrent.Callable<V> eval = () -> c.compute(arg);
                java.util.concurrent.FutureTask<V> ft = new java.util.concurrent.FutureTask<>(eval);
                f = cache.putIfAbsent(arg, ft);
                if (f == null) { f = ft; ft.run(); }
            }
            try { return f.get(); }
            catch (java.util.concurrent.CancellationException e) { cache.remove(arg, f); }
        }
    }
}

Conclusion

A robust local cache should address data structure choice, entry limit, eviction strategy, TTL handling, thread safety, optional persistence, and a blocking mechanism for expensive value creation. The interfaces and code snippets above illustrate a concrete way to build such a cache.