Spring Boot + Caffeine: Multi-Level Cache Architecture, Consistency & Load Tests

1. Scenario and Architecture Trade-offs

In long-running microservices, Redis often becomes the most expensive single point. All traffic hits Redis, concentrating network overhead, serialization latency, and connection-pool contention. A burst or network jitter can trigger JedisConnectionException and drag down the gateway. Pulling hot data into application memory is the lowest-cost, fastest-acting mitigation.

Local cache consumes single-node memory in exchange for nanosecond read latency and zero network I/O . However, it is not a Redis replacement: nodes cannot share data, and strong consistency is fundamentally impossible. The mainstream production pattern is therefore L1(Caffeine) + L2(Redis) + DB: Caffeine absorbs 80–90% of instantaneous reads, Redis provides cross-node state sharing and fallback, and DB remains the source of truth. The core logic is straightforward: trade millisecond-level temporary inconsistency for overall system throughput and survival .

The following sections walk through integration, read/write paths, consistency guarantees, and production tuning using Spring Boot 3.x / Java 17+.

2. Caffeine Internals and Configuration Details

Caffeine’s performance stems from the Window TinyLFU algorithm. It separates recent access from long-term frequency: new entries enter a Window area; when full, they compete with low-frequency entries in the Main area based on weight, with the winner staying. Combined with Count-Min Sketch for frequency estimation, memory overhead stays low. In practice, hit rates exceed Guava’s by 15%+ under equal heap space.

expireAfterWrite vs refreshAfterWrite

Caffeine.newBuilder()
    .maximumSize(10_000)
    .expireAfterWrite(Duration.ofMinutes(30))
    .refreshAfterWrite(Duration.ofMinutes(10))
    .buildAsync(loader);

expireAfterWrite

: evicts immediately on expiry. Next request blocks synchronously while reloading. Under load, the reload thread pool saturates instantly. refreshAfterWrite (mandatory in production): returns stale value after expiry, then refreshes asynchronously in background. Requests never block, avoiding the “mass expiry + synchronous DB query” avalanche. Note: on first cache miss, refreshAfterWrite still blocks once until initial load completes; subsequent reads benefit from async refresh.

Metrics Exposure

CacheStats

fields — hitCount(), missCount(), evictionCount(), averageLoadPenalty() — are the core monitoring dashboard. Spring Boot 3.x includes Micrometer; registering a CacheMetricsRegistrar pushes metrics to Prometheus. Don’t just watch hit rate; a spike in averageLoadPenalty usually signals slow source SQL or downstream service, and should be investigated immediately.

3. Spring Cache Integration: Annotations Are Convenient, But Don’t Trust Them Blindly

spring-boot-starter-cache

provides a unified abstraction; wiring Caffeine is simple.

3.1 Annotation-Driven (Lightweight Scenarios)

@Configuration
public class CacheConfig {
    @Bean
    public CacheManager cacheManager() {
        CaffeineCacheManager manager = new CaffeineCacheManager();
        manager.setCaffeine(Caffeine.newBuilder()
            .maximumSize(5000)
            .expireAfterAccess(5, TimeUnit.MINUTES)
            .recordStats());
        return manager;
    }
}

Used with @Cacheable:

@Cacheable(value = "user_profile", key = "#userId", sync = true)
public UserProfile getUser(String userId) {
    return userRepo.findById(userId).orElse(null);
}

sync = true

adds a key-level lock at the Spring layer (backed by ConcurrentHashMap segments), effectively preventing cache stampede. However, annotations have pitfalls: SpEL expressions are hard to debug, complex key concatenation breeds bugs, and TTL can only be configured globally, making per-domain differentiation cumbersome.

3.2 Manual API (Production Recommended)

For multi-tenant isolation, dynamic TTL, or chaining L1/L2 reads/writes, define the cache bean directly:

@Bean("userCache")
public Cache<String, UserProfile> userCache() {
    return Caffeine.newBuilder()
        .maximumSize(2000)
        .refreshAfterWrite(1, TimeUnit.MINUTES)
        .buildAsync(key -> loadFromDBOrL2(key))
        .synchronous();
}

Holding the Cache instance manually puts loading logic, exception fallback, and null handling fully under your control. Production architectures almost always take this route for flexibility and debuggability.

4. Multi-Level Cache Read/Write Path Design

Stacking code isn’t enough; read and write paths must be nailed down.

4.1 Read Path

Check L1; return immediately on hit.

L1 miss → check L2 (Redis). On hit, write back to L1 and return.

L2 miss → query DB. On hit, dual-write to both L2 and L1.

DB miss → cache a short-TTL empty marker (cache penetration guard).

4.2 Write Path (Invalidation Over Update)

Never touch L1 on writes. Update DB, then publish an invalidation event via message middleware (e.g., invalidate:cacheKey). Each node subscribes and calls cache.invalidate(key) locally.

sequenceDiagram
    Client->>App: Update request
    App->>DB: Commit transaction
    App->>MQ/RedisPubSub: Send invalidation event
    MQ/RedisPubSub->>App-Node1: Consume, evict local Caffeine
    MQ/RedisPubSub->>App-Node2: Consume, evict local Caffeine

Why not write-through directly to local cache? Nodes cannot guarantee ordering or atomicity. The industry’s most stable approach is write DB + async broadcast invalidation , sacrificing a tiny consistency window for absolute write-path stability. L1 retains only the hottest data with TTL of 1–5 minutes; L2 holds full dataset with TTL of 30+ minutes.

5. Handling Consistency

Cross-node inconsistency is unavoidable in multi-level caching. Compressing the window to under 500 ms requires a combination of techniques.

5.1 Message-Driven Invalidation (Primary)

Redis Keyspace Notifications work but can lose messages during network blips or node restarts. Core businesses should use Kafka/RabbitMQ, attaching version and timestamp to each message. Consumers implement eventual-consistency retries; local handling must be idempotent — ignore if key already deleted.

5.2 Version Stamp Fallback

Add updated_at or version to business tables. On read, if local data’s version is lower than DB’s, trigger cache.refresh(key) to pull fresh data. Combined with refreshAfterWrite ’s async mechanism, requests don’t block while data gradually converges.

5.3 Soft Delete Coordination

Logically deleted records ( is_deleted=1) must emit invalidation events synchronously. Query layer must not cache such data . If caching is unavoidable, store a tombstone object with an extremely short TTL; otherwise “ghost caches” cause repeated DB penetration, a problem that often hides during load tests.

6. Load Test Records and Memory/GC Tuning

Last week on an 8C16G test machine (JDK 17, G1 GC), dataset of 100k product configs, Zipf-distributed requests (20% keys carry 80% traffic), wrk at 500 concurrency for two minutes.

Direct DB: connection pool saturated, QPS ~1.8k, P99 >400 ms, CPU waiting on I/O.

Pure Redis: QPS ~14k, but network serialization overhead visible, P99 ~45 ms, higher GC frequency.

Caffeine only: QPS ~48k, P99 ~2 ms. Heap grew ~600 MB, but GC stable — ~8 Young GC/min.

L1+L2 combined: QPS ~42k, P99 ~3.8 ms, heap ~1.6 GB. Slightly lower raw numbers than pure L1, but far more stable during node scaling and Redis hiccups — the highest comprehensive ROI.

Memory and GC Tuning in Practice

maximumSize

controls entry count , not bytes. Large values can still OOM. Strongly recommend configuring a Weigher using key.length + value.serializedSize() and setting maximumWeight to reflect actual heap usage. Keep weight limit under 30% of available heap.

Caffeine uses LongAdder for stats — allocation-friendly for G1. Do not enable weakValues ; production trial showed references scanned too aggressively, doubling Young GC frequency and collapsing hit rate. Stick to strong references with strict capacity control.

Monitor evictionCount and hitRate curves. Sudden eviction spike with stable hit rate = normal churn. Both deteriorating = key design too scattered or capacity too small.

7. Common Production Scenarios

7.1 Hot Key Protection

Single-key burst reads can overwhelm a single core. Combine Resilience4j’s RateLimiter for local queuing: limit concurrent loads per key to 3–5, excess requests degrade or wait for first batch. Add 10% random jitter to TTL to prevent synchronized expiry avalanches.

7.2 Startup Warmup

Cold start leaves L1 empty, flooding L2/DB. Use @EventListener(ApplicationReadyEvent.class) with CompletableFuture to asynchronously fetch core configs and top hot keys. Don’t block startup; log a marker when done.

7.3 Dynamic Config Hot Reload

Listen to Nacos/Apollo for cache.size / cache.ttl changes. Caffeine 3.x supports runtime parameter adjustment:

@NacosConfigListener(dataId = "cache-config")
public void onCacheConfigChange(String config) {
    int newSize = parseSize(config);
    caffeine.policy().eviction().ifPresent(p -> p.setMaximum(newSize));
}

Capacity scales elastically — shrink off-peak, expand peak — without restart. Adjust gradually; sudden large jumps trigger frequent evictions.

8. Pitfalls and Troubleshooting

8.1 Memory Leak Illusions

Setting maximumSize doesn’t guarantee no OOM. If keys/values hold unreleased resources (unclosed InputStream, static large map references) or Weigher returns 0/negative, Caffeine cannot enforce limits. Diagnose with jmap -histo:live or Arthas dashboard; correlate evictionWeight with heap growth. Periodic heap dumps + Eclipse MAT Dominator Tree reveal the culprit instantly.

8.2 Serialization and Key Stability

Spring Cache defaults to JDK serialization — bulky and slow. Production standardizes on Jackson JSON or Kryo. Key hashCode must be stable . Never use Object[] or unordered Set directly as keys; hash changes invalidate cache silently. In SpEL, explicitly stringify: key = "#id.toString()". When upgrading cache structure, use canary dual-write or version-prefix isolation; deserialization errors can crash the entire node.

8.3 Concurrent Cache Penetration

@Cacheable(sync=true)

uses Spring’s key-granularity lock. Manual LoadingCache.get(key) leverages Caffeine’s built-in per-key lock, natively preventing stampede. For DB misses, always cache Optional.empty() or a custom null object with 30–60 second TTL; otherwise every miss hits DB, and slow-query logs will teach you the hard way.

9. Deployment Checklist

Size capacity correctly : don’t guess maximumSize. Analyze a week of access logs, compute Top N hot set size, cap weight at 30% of heap.

Prioritize consistency chain reliability : DB write → MQ publish → node consume → L1 evict. Message persistence and consumer idempotency must be rock-solid. Performance optimization comes after.

Observability end-to-end : push hitRate, loadSuccessCount, evictionCount to Grafana. Alert when hit rate drops below 60% — usually coarse key granularity or missing warmup.

Keep a kill switch : if L1 eviction rate spikes abnormally or L2 connection pool alarms, flip a config-center flag to disable L1 instantly, falling back to L2 or DB. Survive first, tune later.

Don’t chase absolute consistency : accepting brief inconsistency is an architectural prerequisite. Define data boundaries clearly, control load granularity, and the system naturally withstands high concurrency.

The essence of local caching is not “stuff all data into memory” but “use the smallest memory footprint to precisely intercept the hottest traffic wave”. Master the internals, straighten the multi-level paths, harden monitoring, and production runs without 3 AM alerts.