Understanding the Design and Implementation of Caffeine Cache

Caffeine is a high‑performance Java caching library that implements a fixed‑size eviction policy using a ConcurrentHashMap to store entries and three LRU deques—window, probation, and protected—to manage element lifecycles. The window region holds newly added items, probation holds candidates for promotion, and protected stores frequently accessed entries.

Eviction relies on the TinyLFU algorithm, which employs a Count‑Min Sketch to estimate access frequencies with about 93.75% accuracy while using minimal memory. Write and read operations are buffered in multi‑producer, single‑consumer (MPSC) structures: WriteBuffer holds AddTask and UpdateTask objects that update weights, move nodes between regions, and trigger asynchronous maintenance via a PerformCleanupTask submitted to an executor.

The core API methods are V put(K key, V value) and @Nullable V getIfPresent(Object key, boolean recordStats) . put inserts an entry into the map, creates an AddTask , and schedules maintenance; getIfPresent reads directly from the map, updates access time, and enqueues a read task for later processing. Both methods interact with the buffers and invoke afterWrite or afterRead to maintain consistency.

Finally, Caffeine’s adaptive “climb” phase monitors hit‑rate samples and dynamically adjusts the sizes of the window and protected regions. If the hit rate rises, the window expands; if it falls, the window shrinks and the protected region grows, ensuring the cache balances recency and frequency for optimal performance in concurrent environments.