From Redis to Caffeine: iQIYI’s Journey Through Java Caching Strategies

Background and iQIYI’s Cache Evolution

The author attended an iQIYI technical salon and summarized the company’s cache development across five stages, illustrated with diagrams. Stage 1 used message‑queue‑driven data sync to Redis, offering fast updates but risking a cache avalanche if Redis failed. Stages 2‑3 introduced an in‑process Java Map (Guava cache) as a first‑level cache with Redis as second‑level, improving resilience but introducing eviction and hit‑rate challenges due to limited JVM memory.

Stage 4 added write‑after‑refresh to Guava Cache, mitigating stale data but still lacking real‑time refresh. Stage 5 extended Guava with asynchronous external refresh via Redis‑based notifications, allowing Java services to invalidate caches cooperatively.

Primitive Database Lookup (Pre‑Cache Era)

When traffic is low, direct database or file reads suffice, providing simplicity and meeting business needs without any caching layer.

HashMap and LRUMap in Java

As request volume grows, developers resort to HashMap or ConcurrentHashMap for in‑process caching. While fast, HashMap lacks eviction, causing unbounded memory growth. It remains useful for static data such as reflective method lookups where eviction isn’t required.

To address unbounded growth, a custom LRUMap can be built by extending LinkedHashMap and overriding removeEldestEntry. This provides a simple least‑recently‑used eviction policy with modest implementation cost.

Guava Cache: Solving LRUMap Limitations

Guava Cache introduces segment‑based locking (similar to ConcurrentHashMap) to reduce lock contention, configurable concurrencyLevel, and automatic eviction based on size, write‑after‑write, and access‑after‑read policies.

5.1 Lock Competition

Guava divides entries into segments, each protected by its own lock. The segment count (default 4) ensures at least ten entries per segment, balancing contention and random eviction risk.

5.2 Expiration Policies

Guava supports expireAfterWrite and expireAfterAccess. Expiration is performed lazily during reads/writes, avoiding a dedicated cleanup thread and global locking.

5.3 Automatic Refresh

Guava can automatically reload entries by providing a CacheLoader that recomputes values when they become stale.

5.4 Additional Features

Guava allows weak/soft references for keys and values, and a RemovalListener to react to evictions, expirations, or explicit removals.

RemovalCause records all eviction reasons: explicit removal, replacement, expiration, size‑based eviction, or collection.

Caffeine Cache and the W‑TinyLFU Algorithm

Caffeine builds on Guava’s API but replaces the underlying data structures with a high‑performance design based on the W‑TinyLFU algorithm, which combines LFU frequency tracking with LRU recency.

6.1 W‑TinyLFU Overview

Traditional LFU can retain stale hot items (e.g., a popular TV series after its peak). W‑TinyLFU mitigates this by decaying frequencies over time and using a compact sketch to track access counts.

6.2 Frequency Recording with Count‑Min Sketch

Caffeine’s FrequencySketch uses a Count‑Min Sketch: multiple hash functions map an item to several counters in a long array, incrementing each. The minimum counter value across hashes estimates the true frequency while keeping memory usage low.

6.3 Read‑Write Performance

Caffeine offloads eviction and expiration work to an asynchronous ring‑buffer processed by a ForkJoinPool, dramatically reducing contention compared to Guava’s synchronous approach.

6.4 Data Eviction Queues

Caffeine stores entries in three LRU queues inside a ConcurrentHashMap:

Eden queue (≈1% of capacity) holds newly added entries to protect them from immediate eviction.

Probation queue holds cold entries that are candidates for eviction.

Protected queue (≈80% of remaining capacity) holds frequently accessed entries; items promoted from Probation after a hit.

6.5 Using Caffeine

Developers familiar with Guava can adopt Caffeine with minimal code changes because the APIs are intentionally similar.

Conclusion

The article outlines iQIYI’s cache roadmap—from raw database queries to sophisticated, high‑throughput local caches like Guava and Caffeine—highlighting each technique’s motivations, trade‑offs, and implementation details, while noting that effective caching also requires considerations such as multi‑level designs, synchronization across services, and distributed cache integration.