Evolution of Java Caching at iQIYI: From Database Lookups to Guava and Caffeine

The article begins with a background on iQIYI's cache evolution, outlining five stages: initial database lookups, data synchronization with Redis, in‑process Java HashMap and Guava cache, Guava cache enhancements, and finally the modern Caffeine cache.

Stage 1 – Data Sync with Redis: Data is synchronized to Redis via a message queue; Java applications read directly from Redis, offering fast updates but risking cache avalanche if Redis fails.

Stage 2 & 3 – Java HashMap and Guava Cache: An in‑process cache serves as a first‑level cache, with Redis as second‑level. Code example shows a simple HashMap based service:

public class CustomerService {
    private HashMap<String, String> hashMap = new HashMap<>();
    private CustomerMapper customerMapper;
    public String getCustomer(String name) {
        String customer = hashMap.get(name);
        if (customer == null) {
            customer = customerMapper.get(name);
            hashMap.put(name, customer);
        }
        return customer;
    }
}

Limitations include unbounded memory growth and lack of eviction.

Stage 4 – LRUMap: By extending LinkedHashMap and overriding removeEldestEntry, a simple LRU eviction policy is implemented. Example snippet:

class LRUMap extends LinkedHashMap {
    private final int max;
    private final Object lock;
    public LRUMap(int max, Object lock) {
        super((int)(max * 1.4f), 0.75f, true);
        this.max = max;
        this.lock = lock;
    }
    @Override
    protected boolean removeEldestEntry(Map.Entry eldest) {
        return size() > max;
    }
}

Stage 5 – Guava Cache: Guava addresses lock contention, expiration, and automatic refresh. Sample configuration:

LoadingCache<String, String> cache = CacheBuilder.newBuilder()
    .maximumSize(100)
    .expireAfterWrite(30L, TimeUnit.MILLISECONDS)
    .expireAfterAccess(30L, TimeUnit.MILLISECONDS)
    .refreshAfterWrite(20L, TimeUnit.MILLISECONDS)
    .weakKeys()
    .build(createCacheLoader());

Key Guava features discussed include segment‑based locking, lazy expiration during reads/writes, and removal listeners for eviction analysis.

Caffeine and W‑TinyLFU: The article introduces Caffeine as a successor to Guava, highlighting its use of a Count‑Min Sketch (FrequencySketch) to record access frequencies with limited memory, and its three‑queue architecture (Eden, Probation, Protected) implementing the W‑TinyLFU policy. Performance graphs illustrate Caffeine’s superior hit‑rate and throughput compared to Guava.

Usage example mirrors Guava’s API, making migration straightforward:

Cache<String, String> cache = Caffeine.newBuilder()
    .expireAfterWrite(1, TimeUnit.SECONDS)
    .expireAfterAccess(1, TimeUnit.SECONDS)
    .maximumSize(10)
    .build();
cache.put("hello", "hello");

The article concludes by encouraging readers to explore multi‑level caching, distributed caches, and deeper source‑code analysis of Guava and Caffeine.