Why Simple Cache Add‑On Fails: Master These Patterns to Avoid Performance Disasters

Cache design is more than just inserting a Map or Redis instance; the choice of pattern, eviction policy, concurrency control, and distributed consistency determines whether a system speeds up or crashes.

Cache Access Patterns

1. Cache‑Aside – the application reads the cache first and falls back to the data source on miss. Code example:

package com.icoderoad.cache;

import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

class CacheAsideCache {
    private final Map<String, String> cache = new ConcurrentHashMap<>();
    private final DataSource dataSource;

    public CacheAsideCache(DataSource dataSource) {
        this.dataSource = dataSource;
    }

    public String get(String key) {
        // check cache
        String value = cache.get(key);
        if (value != null) {
            return value;
        }
        // miss – load from source
        value = dataSource.load(key);
        if (value != null) {
            cache.put(key, value);
        }
        return value;
    }
}

Advantages: simple and flexible. Drawbacks: no strong coupling with the database, leading to stale data, cache breakdown, and dirty reads. Suitable for read‑heavy, write‑light workloads where temporary inconsistency is acceptable.

2. Write‑Through – writes go to both cache and database synchronously.

package com.icoderoad.cache;

import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

class WriteThroughCache {
    private final Map<String, String> cache = new ConcurrentHashMap<>();
    private final DataSource dataSource;

    public WriteThroughCache(DataSource dataSource) {
        this.dataSource = dataSource;
    }

    public void put(String key, String value) {
        cache.put(key, value);
        dataSource.save(key, value);
    }

    public String get(String key) {
        return cache.get(key);
    }
}

Provides strong consistency and eliminates dirty reads, but each write incurs two operations, reducing write throughput. Ideal for scenarios demanding strict correctness, such as account balances or order status.

3. Write‑Back – writes update the cache first; the database is updated asynchronously.

package com.icoderoad.cache;

import java.util.Map;
import java.util.concurrent.*;

class WriteBackCache {
    private final Map<String, String> cache = new ConcurrentHashMap<>();
    private final DataSource dataSource;
    private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(2);

    public void put(String key, String value) {
        cache.put(key, value);
        scheduler.schedule(() -> dataSource.save(key, value), 5, TimeUnit.SECONDS);
    }
}

Offers the highest write performance, suitable for high‑concurrency write workloads, but data not yet persisted can be lost on crash. Use when occasional loss is tolerable (e.g., logs, statistics).

Eviction Strategies

LRU (Least Recently Used) – retains recently accessed entries.

class LRUCache<K, V> extends LinkedHashMap<K, V> {
    private final int maxSize;
    public LRUCache(int maxSize) {
        super(16, 0.75f, true);
        this.maxSize = maxSize;
    }
    @Override
    protected boolean removeEldestEntry(Map.Entry<K, V> eldest) {
        return size() > maxSize;
    }
}

Best for hot‑spot data such as product detail pages.

LFU (Least Frequently Used) – evicts entries with the lowest access count.

class LFUCache<K, V> {
    private final Map<K, V> cache = new HashMap<>();
    private final Map<K, Integer> frequency = new HashMap<>();
    private final int maxSize;
    public LFUCache(int maxSize) { this.maxSize = maxSize; }
    public V get(K key) {
        V value = cache.get(key);
        if (value != null) {
            frequency.put(key, frequency.getOrDefault(key, 0) + 1);
        }
        return value;
    }
    public void put(K key, V value) {
        if (cache.size() >= maxSize && !cache.containsKey(key)) {
            evictLeastFrequent();
        }
        cache.put(key, value);
        frequency.put(key, frequency.getOrDefault(key, 0) + 1);
    }
    private void evictLeastFrequent() { /* remove entry with smallest frequency */ }
}

Fits systems with stable access patterns, like recommendation engines.

TTL (Time‑to‑Live) – entries expire after a configured duration.

class TTLCache<K, V> {
    private final Map<K, CacheEntry<V>> cache = new ConcurrentHashMap<>();
    private final long ttlMillis;
    public TTLCache(long ttlMillis) { this.ttlMillis = ttlMillis; }
    public V get(K key) {
        CacheEntry<V> entry = cache.get(key);
        if (entry == null || entry.isExpired()) {
            cache.remove(key);
            return null;
        }
        return entry.value();
    }
    public void put(K key, V value) {
        long expiry = System.currentTimeMillis() + ttlMillis;
        cache.put(key, new CacheEntry<>(value, expiry));
    }
    static class CacheEntry<V> {
        private final V value;
        private final long expiryTime;
        CacheEntry(V v, long e) { this.value = v; this.expiryTime = e; }
        boolean isExpired() { return System.currentTimeMillis() > expiryTime; }
        V value() { return value; }
    }
}

Useful when data freshness is less critical.

Concurrency Control

High‑concurrency environments require thread‑safe caches.

ConcurrentHashMap – lock‑free, suitable for most cases.

synchronizedMap – simple wrapper, but incurs higher contention.

Read‑Write Lock – read‑heavy workloads benefit from separate read/write locks.

class ReadWriteCache<K, V> {
    private final Map<K, V> cache = new HashMap<>();
    private final ReadWriteLock lock = new ReentrantReadWriteLock();
    public V get(K key) {
        lock.readLock().lock();
        try { return cache.get(key); }
        finally { lock.readLock().unlock(); }
    }
    public void put(K key, V value) {
        lock.writeLock().lock();
        try { cache.put(key, value); }
        finally { lock.writeLock().unlock(); }
    }
}

Distributed Cache Considerations

Moving from single‑node to distributed caches multiplies complexity.

Invalidation Broadcast – each node must receive a notification to evict stale entries.

class DistributedCache {
    private final LocalCache localCache;
    public DistributedCache(LocalCache lc) { this.localCache = lc; }
    public void invalidate(String key) {
        localCache.remove(key);
        notifyOtherNodes(key);
    }
    private void notifyOtherNodes(String key) { /* send invalidation message */ }
}

Consistency Models – trade‑off between latency and correctness:

Strong consistency guarantees absolute correctness but adds latency.

Eventual consistency offers better performance at the cost of short‑term divergence.

There is no universally perfect solution; the model must match business requirements.

Network Overhead – minimize remote calls by combining local cache with distributed cache, pre‑warming hot data, and batching operations.

Production‑Ready Cache Composition

A practical cache combines multiple capabilities:

Read/write strategy (Cache‑Aside, Write‑Through, Write‑Back).

Eviction policy (LRU, LFU, TTL).

Concurrency control (lock‑free or lock‑based).

Distributed consistency mechanism.

class ProductionCache<K, V> {
    private final ConcurrentHashMap<K, CacheEntry<V>> cache = new ConcurrentHashMap<>();
    private final int maxSize;
    private final long ttlMillis;
    private final EvictionStrategy evictionStrategy;
    public ProductionCache(int maxSize, long ttlMillis, EvictionStrategy es) {
        this.maxSize = maxSize;
        this.ttlMillis = ttlMillis;
        this.evictionStrategy = es;
    }
    public V get(K key) {
        CacheEntry<V> e = cache.get(key);
        if (e == null) return null;
        if (e.isExpired()) { cache.remove(key); return null; }
        e.updateAccessTime();
        return e.value();
    }
    public void put(K key, V value) {
        if (cache.size() >= maxSize && !cache.containsKey(key)) {
            evictionStrategy.evict(cache);
        }
        long expiry = System.currentTimeMillis() + ttlMillis;
        cache.put(key, new CacheEntry<>(value, expiry));
    }
}

Practical Decision Checklist

Read‑heavy & write‑light → prefer Cache‑Aside.

Strong consistency needed → choose Write‑Through.

High write pressure → consider Write‑Back.

Hotspot data → use LRU.

Stable access frequency → use LFU.

Data with a clear lifetime → use TTL.

Ensure memory limits, guard against cache breakdown or avalanche, and evaluate need for distributed synchronization.

Conclusion

Cache is not merely an acceleration component; it is a full‑stack design discipline. Understanding patterns, eviction policies, concurrency mechanisms, and distributed consistency determines whether a system gains stable performance or suffers instability.