How Caffeine’s ReadBuffer Works: Deep Dive into getIfPresent and Eviction Mechanics

getIfPresent

After a basic understanding of the put method, we continue to explore the getIfPresent method.

public class TestReadSourceCode {
    @Test
    public void doRead() {
        // read constructor
        Cache<String, String> cache = Caffeine.newBuilder()
                .maximumSize(10_000)
                .build();
        // read put
        cache.put("key", "value");
        // read get
        cache.getIfPresent("key");
    }
}

The corresponding source code with comments is shown below:

abstract class BoundedLocalCache<K, V> extends BLCHeader.DrainStatusRef implements LocalCache<K, V> {
    final ConcurrentHashMap<Object, Node<K, V>> data;
    final Buffer<Node<K, V>> readBuffer;
    @Override
    public @Nullable V getIfPresent(Object key, boolean recordStats) {
        // Directly obtain element from ConcurrentHashMap
        Node<K, V> node = data.get(nodeFactory.newLookupKey(key));
        if (node == null) {
            // Update miss statistics
            if (recordStats) {
                statsCounter().recordMisses(1);
            }
            // If drainStatus is REQUIRED, schedule maintenance
            if (drainStatusOpaque() == REQUIRED) {
                scheduleDrainBuffers();
            }
            return null;
        }
        V value = node.getValue();
        long now = expirationTicker().read();
        // Check expiration or collection
        if (hasExpired(node, now) || (collectValues() && (value == null))) {
            if (recordStats) {
                statsCounter().recordMisses(1);
            }
            scheduleDrainBuffers();
            return null;
        }
        // Ensure not computing async
        if (!isComputingAsync(node)) {
            @SuppressWarnings("unchecked")
            K castedKey = (K) key;
            setAccessTime(node, now);
            tryExpireAfterRead(node, castedKey, value, expiry(), now);
        }
        // Post‑read processing
        V refreshed = afterRead(node, now, recordStats);
        return (refreshed == null) ? value : refreshed;
    }
}

In the getIfPresent method, we have already introduced scheduleDrainBuffers; the final step focuses on afterRead, which handles post‑read operations.

abstract class BoundedLocalCache<K, V> extends BLCHeader.DrainStatusRef implements LocalCache<K, V> {
    final Buffer<Node<K, V>> readBuffer;
    @Nullable V afterRead(Node<K, V> node, long now, boolean recordHit) {
        // Update hit statistics
        if (recordHit) {
            statsCounter().recordHits(1);
        }
        // If skipReadBuffer is false, offer to readBuffer
        boolean delayable = skipReadBuffer() || (readBuffer.offer(node) != Buffer.FULL);
        // Determine if delayed processing is needed
        if (shouldDrainBuffers(delayable)) {
            scheduleDrainBuffers();
        }
        // Refresh if needed
        return refreshIfNeeded(node, now);
    }
    boolean skipReadBuffer() {
        // Fast‑path check for skip
        return fastpath() && frequencySketch().isNotInitialized();
    }
    boolean shouldDrainBuffers(boolean delayable) {
        switch (drainStatusOpaque()) {
            case IDLE: return !delayable;
            case REQUIRED: return true;
            case PROCESSING_TO_IDLE:
            case PROCESSING_TO_REQUIRED: return false;
            default: throw new IllegalStateException("Invalid drain status: " + drainStatus);
        }
    }
}

The ReadBuffer is a multiple‑producer/single‑consumer (MPSC) buffer that uses a segmented design to reduce contention. Its actual type becomes BoundedBuffer when conditions are met, as illustrated below:

The Buffer interface is described as a MPSC buffer that rejects new elements when full and does not guarantee FIFO or LIFO ordering.

A multiple‑producer / single‑consumer buffer that rejects new elements if it is full or fails spuriously due to contention. Unlike a queue or stack, a buffer does not guarantee FIFO or LIFO ordering.

The StripedBuffer implementation uses segmenting and CAS to achieve efficient concurrent writes. Its offer method hashes the thread probe to a segment and attempts to insert the element, expanding or retrying as needed.

abstract class StripedBuffer<E> implements Buffer<E> {
    volatile Buffer<E> @Nullable[] table;
    @Override
    public int offer(E e) {
        long z = mix64(Thread.currentThread().getId());
        int increment = ((int) (z >>> 32)) | 1;
        int h = (int) z;
        int mask;
        int result;
        Buffer<E> buffer;
        Buffer<E>[] buffers = table;
        if (buffers == null || (mask = buffers.length - 1) < 0 ||
            (buffer = buffers[h & mask]) == null ||
            !(uncontended = ((result = buffer.offer(e)) != Buffer.FAILED))) {
            return expandOrRetry(e, h, increment, uncontended);
        }
        return result;
    }
}

The expandOrRetry method handles initialization, resizing, creation of new buffers, and contention, as shown in the source code.

final int expandOrRetry(E e, int h, int increment, boolean wasUncontended) {
    int result = Buffer.FAILED;
    boolean collide = false;
    for (int attempt = 0; attempt < ATTEMPTS; attempt++) {
        Buffer<E>[] buffers;
        Buffer<E> buffer;
        int n;
        if ((buffers = table) != null && (n = buffers.length) > 0) {
            if ((buffer = buffers[(n - 1) & h]) == null) {
                if (tableBusy == 0 && casTableBusy()) {
                    boolean created = false;
                    try {
                        Buffer<E>[] rs;
                        int mask, j;
                        if ((rs = table) != null && (mask = rs.length) > 0 &&
                            (rs[j = (mask - 1) & h] == null)) {
                            rs[j] = create(e);
                            created = true;
                        }
                    } finally {
                        tableBusy = 0;
                    }
                    if (created) {
                        result = Buffer.SUCCESS;
                        break;
                    }
                    continue;
                }
                collide = false;
            } else if (!wasUncontended) {
                wasUncontended = true;
            } else if ((result = buffer.offer(e)) != Buffer.FAILED) {
                break;
            } else if (n >= MAXIMUM_TABLE_SIZE || table != buffers) {
                collide = false;
            } else if (!collide) {
                collide = true;
            } else if ((tableBusy == 0) && casTableBusy()) {
                try {
                    if (table == buffers) {
                        table = Arrays.copyOf(buffers, n << 1);
                    }
                } finally {
                    tableBusy = 0;
                }
                collide = false;
                continue;
            }
            h += increment;
        } else if ((tableBusy == 0) && (table == buffers) && casTableBusy()) {
            boolean init = false;
            try {
                Buffer<E>[] rs = new Buffer[1];
                rs[0] = create(e);
                table = rs;
                init = true;
            } finally {
                tableBusy = 0;
            }
            if (init) {
                result = Buffer.SUCCESS;
                break;
            }
        }
    }
    return result;
}

The ReadBuffer is an MPSC buffer that partitions the buffer into multiple segments, reducing contention and using CAS for safe concurrent writes. It does not guarantee FIFO or LIFO ordering.

The maintenance method orchestrates the processing of read and write buffers, key/value references, expiration, eviction, and the “climb” operation that dynamically adjusts window and protected sizes based on hit rate.

void maintenance(@Nullable Runnable task) {
    setDrainStatusRelease(PROCESSING_TO_IDLE);
    try {
        drainReadBuffer();
        drainWriteBuffer();
        if (task != null) {
            task.run();
        }
        drainKeyReferences();
        drainValueReferences();
        expireEntries();
        evictEntries();
        climb();
    } finally {
        if ((drainStatusOpaque() != PROCESSING_TO_IDLE) || !casDrainStatus(PROCESSING_TO_IDLE, IDLE)) {
            setDrainStatusOpaque(REQUIRED);
        }
    }
}

The climb method adapts the eviction policy toward an optimal recency/frequency configuration by adjusting window and protected capacities based on hit‑rate changes.

void climb() {
    if (!evicts()) return;
    determineAdjustment();
    demoteFromMainProtected();
    long amount = adjustment();
    if (amount == 0) return;
    if (amount > 0) {
        increaseWindow();
    } else {
        decreaseWindow();
    }
}

Overall, Caffeine’s cache combines a TinyLFU eviction policy with segmented MPSC buffers, dynamic window/protected sizing, and a maintenance cycle that balances recency and frequency for high‑performance caching in concurrent Java applications.