Master Multi‑Level Cache Design: From Redis to JVM with Caffeine

Article Overview

Keywords: high performance, cache, Redis, I/O. Introduction: From everyday scenarios to real‑world projects, this piece offers thoughts on cache design to serve as a stepping stone for technical advancement.

Life Scenario

On a Sunday morning, two friends decide to eat hot pot. At the restaurant a self‑service utensil cabinet creates unnecessary door opening/closing overhead. Placing utensils outside lets customers grab them directly, avoiding frequent I/O.

Customers take utensils from the external place (cache).

If external utensils are empty, staff replenish from the cabinet (load from disk to memory, avoiding frequent I/O).

If staff are busy, customers take directly from the cabinet (disk).

Cache Design Model

Abstracted simple cache model.

Definitions

Disk

What: Large persistent storage for long‑term data.

How: Data stored on disk remains after power off.

Example: A wardrobe for infrequently used items.

Memory

What: Short‑term memory of a computer, holding currently used programs and data.

How: Fast read/write while the machine runs; cleared on shutdown.

Example: A desk workspace where you place items while working.

Cache

What: Smaller, faster storage for temporary holding of frequently accessed data.

How: Keeps hot data for quick retrieval.

Example: A sticky note on your desk with the most used information.

I/O

What: Data transfer between the computer and the external world (input and output).

How: Reading files from disk (input) or displaying data on a screen (output).

Example: Reading a book (input) or writing your thoughts on paper (output).

Summary

We now have a conceptual understanding of cache. In real projects traffic can range from tens of thousands to billions of requests. Cache design aims to avoid frequent I/O, improve throughput, and enhance system performance. Next we will explore concrete cache designs used in practice.

Traditional Cache Design

Typical flow: request reaches Tomcat, queries Redis, if missed then queries the database.

Problems:

Tomcat becomes the performance bottleneck.

When Redis cache expires it puts pressure on the database.

Multi‑Level Cache

What is multi‑level cache?

Multi‑level cache adds caches at each processing stage to reduce server pressure.

Workflow:

Browser reads local cache for static resources.

For dynamic requests, the client contacts the server.

When the request reaches Nginx, Nginx reads its local cache first.

If Nginx cache misses, it queries Redis directly (bypassing Tomcat).

If Redis misses, the request reaches Tomcat and checks the JVM process cache.

If JVM cache misses, the database is queried.

In this architecture Nginx is no longer a simple reverse proxy but a business Web server that contains cache‑query logic.

Key Points

Browser local cache

Nginx local cache (OpenResty + Lua)

Redis cache

JVM process cache

We will focus on JVM process cache and Redis cache.

JVM Process Cache (Caffeine)

Caffeine introduction

Caffeine is a Java‑8 based high‑performance local cache library; Spring's cache abstraction uses it.

Dependency:

<dependency>
  <groupId>com.github.ben-manes.caffeine</groupId>
  <artifactId>caffeine</artifactId>
</dependency>

Basic usage:

@Test
void testBasicOps() {
    // create cache object
    Cache<String, String> cache = Caffeine.newBuilder()
        .expireAfterWrite(Duration.ofSeconds(10)) // cache expires 10 seconds after last write
        .build();

    // store data
    cache.put("code", "码易有道");

    // get data, null if absent
    String val = cache.getIfPresent("code_01"); // null

    // get data, load from DB if absent
    String dbVal = cache.get("ping", key -> {
        // query DB here
        return "pong";
    });
}

Real‑world case requirements:

Add cache for product lookup by id; on miss query the database.

Initial cache size 100.

Maximum size 10,000.

Configuration:

@Configuration
public class CaffeineConfig {
    @Bean
    public Cache<Long, Item> itemCache() {
        return Caffeine.newBuilder()
            .initialCapacity(100)
            .maximumSize(10_000)
            .build();
    }
}

Business logic:

@GetMapping("/{id}")
public Item findById(@PathVariable("id") Long id) {
    return itemCache.get(id, key -> itemService.query()
        .ne("status", 2)
        .eq("id", key)
        .one());
}

Demo shows first request hits the database, second request hits the local cache.

Redis Cache

How fast is Redis?

Official benchmark reports up to ~100,000 QPS.

Basic usage flow diagram:

Implementation steps:

Import dependency.

Configure connection.

Write code to query and populate cache.

@Autowired
private StringRedisTemplate redisTemplate;
private static final ObjectMapper MAPPER = new ObjectMapper();

@GetMapping("/redis/{id}")
public String queryRedisItemData(@PathVariable("id") Long id) throws JsonProcessingException {
    String cachedResult = redisTemplate.opsForValue().get("item:id:" + id);
    if (StringUtils.isNotBlank(cachedResult)) {
        return cachedResult; // hit Redis
    } else {
        Item item = itemService.query().ne("status", 2).eq("id", id).one();
        String dbResult = MAPPER.writeValueAsString(item);
        redisTemplate.opsForValue().set("item:id:" + id, dbResult);
        return dbResult; // miss Redis, then cache
    }
}

Demo shows first request queries the database, second request retrieves data from Redis.

Cache Warm‑up

Cold start occurs when Redis is empty at service startup. Warm‑up pre‑loads hot data.

Option 1 – initialize in a bean implementing InitializingBean:

@Component
public class RedisHandler implements InitializingBean {
    @Autowired private StringRedisTemplate redisTemplate;
    @Autowired private IItemService itemService;
    private static final ObjectMapper MAPPER = new ObjectMapper();

    @Override
    public void afterPropertiesSet() throws Exception {
        List<Item> itemList = itemService.list();
        for (Item item : itemList) {
            String json = MAPPER.writeValueAsString(item);
            redisTemplate.opsForValue().set("item:id:" + item.getId(), json);
        }
    }
}

Option 2 – expose an endpoint to trigger loading:

@GetMapping("/loadRedis")
public String loadRedis() throws JsonProcessingException {
    try {
        List<Item> itemList = itemService.list();
        for (Item item : itemList) {
            String json = MAPPER.writeValueAsString(item);
            redisTemplate.opsForValue().set("item:id:" + item.getId(), json);
        }
    } catch (Exception e) {
        return "loadRedis failed!";
    }
    return "loadRedis success!";
}

Data Synchronization Strategies

Set expiration : simple, but may cause stale data before expiration.

Double write : update DB and cache together; strong consistency but higher coupling.

Async notification : emit an event on DB change; listeners update cache, offering lower coupling with moderate timeliness.

Async Notification Schemes

1) MQ based: business service sends a message, cache service consumes and updates.

2) Canal based: Canal reads MySQL binlog and notifies cache services.

Canal setup (binary log, master‑slave) and sample handler:

@CanalTable("tb_item")
@Component
public class ItemHandler implements EntryHandler<Item> {
    @Autowired private RedisHandler redisHandler;
    @Autowired private Cache<Long, Item> itemCache;

    @Override
    public void insert(Item item) {
        itemCache.put(item.getId(), item);
        redisHandler.saveItem(item);
    }

    @Override
    public void update(Item before, Item after) {
        itemCache.put(after.getId(), after);
        redisHandler.saveItem(after);
    }

    @Override
    public void delete(Item item) {
        itemCache.invalidate(item.getId());
        redisHandler.deleteItemById(item.getId());
    }
}

OpenResty Nginx Local Cache

OpenResty adds a shared dictionary (shdict) that can be accessed by all Nginx workers.

# shared dict definition in nginx.conf
lua_shared_dict item_cache 150m;

Lua usage example:

local item_cache = ngx.shared.item_cache
item_cache:set('key', 'value', 1000) -- expires after 1000 seconds
local val = item_cache:get('key')

Full item.lua script (simplified) shows reading from local cache, falling back to Redis, then HTTP, and finally writing back to the local cache.

Conclusion

When selecting a cache design, consider the specific use case. Options include browser cache for static assets, OpenResty Nginx local cache for dynamic responses, Redis distributed cache for high‑concurrency scenarios, and JVM process cache for in‑process data. Proper cache management, monitoring, and avoiding over‑design are essential for achieving performance and scalability.