14 Painful Spring Boot Cache Pitfalls and How to Build a High‑Availability Architecture

Cache often gives a false sense of safety: high hit rates make you think the system is stable. In reality, the real killers are the interactions between cache, database, traffic spikes, and failure recovery, which are far more complex than a simple key‑value lookup.

A recent e‑commerce promotion pushed the core transaction QPS from the usual 40 k to nearly 150 k. The stack included Spring Boot, Redis cluster, MySQL master‑slave, a message queue, and Kubernetes, yet a series of typical incidents still occurred: 00:03 Product‑detail API P99 latency jumped from 80 ms to 1.8 s; Redis hot‑shard CPU saturated. 00:08 Some hot keys expired simultaneously, causing massive DB‑pool exhaustion. 00:14 Inventory service suffered concurrent write‑back, leading to temporary cache‑DB inconsistency and over‑sell alerts. 00:21 Redis master hiccuped; during Sentinel failover clients retried aggressively, piling up threads. 00:33 Ops cleared the cache to stop the bleeding, pushing the database to the brink of collapse. 01:10 Application throttled non‑core endpoints; DB pressure finally eased.

The article is not a "14‑concept cache quick‑look" but a complete chain of the most common, easily underestimated failure scenarios, explaining why they happen, at what scale, why naive fixes fail, and how to implement them correctly in Spring Boot.

Conclusion: Adding a Redis instance does not finish the cache architecture

Distributed caching must simultaneously address at least four problems:

Shave peak read traffic so the database does not handle every query.

Decouple the write path so a DB change does not instantly ripple through the whole chain.

Isolate partial failures so a Redis outage does not become a DB outage.

Balance data freshness, defining how stale data, staleness duration, and fallback responsibilities are handled.

If a system only implements "if cache miss then query DB and write back", it is merely cache access, not a full cache architecture.

A production‑grade pipeline typically looks like this:

User Request → Gateway → Spring Boot Service → Caffeine (local) → Redis Cluster → MySQL → MQ / Binlog subscription → Rate‑limit / Circuit‑break / Degrade → Monitoring & Alerts

Each layer serves a concrete purpose, not just to look sophisticated:

Local cache handles hot‑key spikes and short Redis outages.

Redis provides shared read load; it is not a strong‑consistency store.

MQ or Binlog subscription guarantees eventual consistency after a cache‑delete failure.

Circuit‑break and rate‑limit bound the impact of cache failures.

Monitoring tells you whether you can see the problem before it escalates.

Scenario 1 – Cache Penetration (non‑existent data floods DB)

Two common triggers:

Malicious requests repeatedly query a non‑existent ID.

Business code generates dirty keys (empty values, wrong tenant prefix, gray‑environment traffic).

Penetration is not just a cache miss; the request misses both cache and DB, forcing a full DB lookup each time. High ratios quickly overwhelm the database.

Spring Boot’s typical fix is "null‑value cache + Bloom filter", but they solve different problems:

Null‑value cache prevents repeated DB lookups for the same missing object.

Bloom filter blocks obviously illegal queries from even reaching the cache layer.

@Configuration
public class BloomFilterConfig {
    @Bean
    public RBloomFilter<String> productBloomFilter(RedissonClient redissonClient) {
        RBloomFilter<String> bloom = redissonClient.getBloomFilter("product:bloom");
        bloom.tryInit(1_000_000L, 0.0001);
        return bloom;
    }
}

@Service
public class ProductQueryService {
    @Autowired private RBloomFilter<String> productBloomFilter;
    @Autowired private RedisTemplate<String, Object> redisTemplate;
    @Autowired private ProductMapper productMapper;

    public ProductDTO getProduct(Long productId) {
        String cacheKey = "product:" + productId;
        if (!productBloomFilter.contains(cacheKey)) return null;
        Object cached = redisTemplate.opsForValue().get(cacheKey);
        if (cached instanceof NullValueMarker) return null;
        if (cached instanceof ProductDTO) return (ProductDTO) cached;
        ProductDTO product = productMapper.selectDtoById(productId);
        if (product == null) {
            redisTemplate.opsForValue().set(cacheKey, new NullValueMarker(), 3, TimeUnit.MINUTES);
            return null;
        }
        redisTemplate.opsForValue().set(cacheKey, product, 30, TimeUnit.MINUTES);
        return product;
    }
}

Two essential points:

The Bloom filter and cache key must share the same naming convention; otherwise you filter one set while querying another.

Null‑value TTL must be short to avoid serving stale "null" after the DB has a new record.

Bloom filters work best when the key set is relatively stable and can be pre‑warmed. If IDs are created extremely frequently or reused after deletion, the false‑positive cost must be re‑evaluated.

Scenario 2 – Cache Breakdown (hot key expires)

Breakdown differs from penetration: the data exists but expires during a traffic spike. Thousands of threads discover the miss simultaneously and all hit the DB, creating a "thundering herd" that can crash the database.

The simplest remedy is a distributed lock:

public ProductDTO getProductWithMutex(Long productId) {
    String cacheKey = "product:" + productId;
    ProductDTO cached = (ProductDTO) redisTemplate.opsForValue().get(cacheKey);
    if (cached != null) return cached;
    RLock lock = redissonClient.getLock("lock:product:" + productId);
    try {
        if (!lock.tryLock(300, 10_000, TimeUnit.MILLISECONDS)) {
            Thread.sleep(50);
            return (ProductDTO) redisTemplate.opsForValue().get(cacheKey);
        }
        ProductDTO doubleCheck = (ProductDTO) redisTemplate.opsForValue().get(cacheKey);
        if (doubleCheck != null) return doubleCheck;
        ProductDTO loaded = productMapper.selectDtoById(productId);
        if (loaded != null) {
            int ttl = 30 + ThreadLocalRandom.current().nextInt(10);
            redisTemplate.opsForValue().set(cacheKey, loaded, ttl, TimeUnit.MINUTES);
        }
        return loaded;
    } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
        return null;
    } finally {
        if (lock.isHeldByCurrentThread()) lock.unlock();
    }
}

In practice, three problems dominate:

Lock wait time too long – application threads pile up.

Lock expires before cache rebuild finishes.

Too many hot keys cause the Redis node itself to become a bottleneck.

For very hot objects, most teams prefer "logical expiration + asynchronous background refresh" instead of making every request wait on a lock. This sacrifices a short window of staleness for stable throughput.

Scenario 3 – Cache Avalanche

An avalanche is not merely "many keys expire at once"; it is a short‑term loss of cache capacity that forces a massive surge of DB traffic, overwhelming downstream services.

Four typical triggers:

All keys share the same TTL.

Redis node failure or network partition makes an entire shard unavailable.

Cache is not pre‑warmed after a deployment; a flood of cold requests hits the DB.

Ops manually clears the cache without accompanying rate‑limit or fallback.

Randomising TTLs only mitigates the first cause. True resilience requires designing the system's behaviour after cache loss:

Rate‑limit incoming traffic.

Provide a local cache fallback that can serve a few seconds of stale data.

Gracefully degrade non‑core fields.

Limit concurrent DB back‑fills.

Scenario 4 – Write‑Side Inconsistency (delete‑then‑update)

The classic mistake is to delete the cache first, then update the DB. If concurrent reads arrive between the two steps, they may repopulate the cache with stale data, causing the cache to hold the old value after the DB commit.

The widely‑adopted Cache‑Aside pattern reverses the order:

Update the DB.

Delete the cache.

@Transactional
public void updateProduct(ProductDTO product) {
    productMapper.updateById(product);
    redisTemplate.delete("product:" + product.getId());
}

Because cache deletion can also fail, production‑grade solutions add two more safeguards:

Main flow: update DB then delete cache immediately.

Compensation flow: publish a delete event to MQ.

Final fallback: Binlog subscription performs a second delete based on the actual commit result.

@Transactional
public void updateProduct(ProductDTO product) {
    productMapper.updateById(product);
    String cacheKey = "product:" + product.getId();
    redisTemplate.delete(cacheKey);
    cacheSyncProducer.sendDelete(cacheKey);
}

@RabbitListener(queues = "cache.delete.queue")
public void onCacheDelete(String cacheKey) {
    redisTemplate.delete(cacheKey);
}

Two subtle failure branches are often missed:

DB transaction rolls back but the delete message has already been sent.

Consumer successfully deletes the cache, but a later cache rebuild occurs after the DB write, re‑introducing stale data.

Therefore, delete‑events must be part of a reliable event system or be driven by Binlog subscription, not assumed to guarantee consistency.

Scenario 5 – BigKey (large values)

BigKey issues surface during replication, migration, deletion, or master‑slave failover. Symptoms include:

Large network payloads increase request latency.

Replication blocks, extending failover time.

Deletion blocks the thread, affecting other requests on the same node.

Memory fragmentation reduces usable memory.

Best practice is to define explicit size thresholds and split large objects into multiple keys based on access patterns:

public void cacheProduct(ProductAggregate agg) {
    String id = String.valueOf(agg.getId());
    redisTemplate.opsForValue().set("product:"+id+":base", agg.getBase(), 30, TimeUnit.MINUTES);
    redisTemplate.opsForValue().set("product:"+id+":price", agg.getPrice(), 5, TimeUnit.MINUTES);
    redisTemplate.opsForValue().set("product:"+id+":media", agg.getMedia(), 60, TimeUnit.MINUTES);
}

public void deleteProductCache(Long productId) {
    redisTemplate.unlink(
        "product:"+productId+":base",
        "product:"+productId+":price",
        "product:"+productId+":media"
    );
}

Splitting should consider:

High‑frequency fields are stored separately to avoid rewriting the whole object on each update.

Very large fields (e.g., images) are isolated so normal queries do not pull them.

Fields that can be updated asynchronously are split to allow partial staleness.

Scenario 6 – Hot Key Hotspot (uneven load)

Even with a healthy cluster, a single slot can become a hotspot, saturating CPU, network, or connections while the cluster’s average metrics look fine.

Typical mitigation is a local cache front‑layer:

private final Cache<String, ProductDTO> localCache = Caffeine.newBuilder()
    .maximumSize(20_000)
    .expireAfterWrite(Duration.ofSeconds(2))
    .build();

public ProductDTO queryHotProduct(Long productId) {
    String key = "product:" + productId;
    ProductDTO local = localCache.getIfPresent(key);
    if (local != null) return local;
    ProductDTO remote = (ProductDTO) redisTemplate.opsForValue().get(key);
    if (remote != null) localCache.put(key, remote);
    return remote;
}

Local cache is not free lunch:

Data is not shared across instances – short‑term inconsistency is inevitable.

Container scaling can cause cache cold‑starts, amplifying load spikes.

After a node migration, stale local entries may linger, skewing memory‑hit ratios.

Thus it fits read‑many/write‑few scenarios such as product details, dictionary data, or recommendation results, but not inventory or balance data that require strong freshness.

Scenario 7 – Concurrent Writes to the Same Key

When multiple service instances write the same cache key, overwrite races occur (e.g., inventory rollback, coupon state correction). If eventual correctness is sufficient, a distributed lock can serialize writes. If high throughput is required, a version number should be introduced.

Lua CAS example:

local key = KEYS[1]
local expectedVersion = tonumber(ARGV[1])
local nextValue = ARGV[2]
local currentVersion = tonumber(redis.call('HGET', key, 'version') or '-1')
if currentVersion ~= expectedVersion then return 0 end
redis.call('HSET', key, 'value', nextValue)
redis.call('HINCRBY', key, 'version', 1)
return 1

This works when the caller can determine the version it is updating; it is unsuitable for unordered consumption where idempotence and ordering must be handled upstream.

Scenario 8 – Cache Warm‑up (cold start)

After a deployment, an empty cache forces the first wave of traffic to hit the DB, often causing the system to jitter. Loading the entire dataset at startup is usually worse, creating massive I/O spikes.

Recommended warm‑up strategy:

Only pre‑warm true hot‑set.

Load asynchronously, in batches, with rate‑limiting.

New instances should first warm their local cache, not flood Redis.

@Component
public class HotProductCacheWarmer {
    @EventListener(ApplicationReadyEvent.class)
    public void warmUp() {
        CompletableFuture.runAsync(() -> {
            List<ProductDTO> hotProducts = productMapper.selectRecentHotProducts(2000);
            for (ProductDTO p : hotProducts) {
                redisTemplate.opsForValue().set("product:"+p.getId(), p,
                    30 + ThreadLocalRandom.current().nextInt(10), TimeUnit.MINUTES);
            }
        });
    }
}

During the release window, temporarily lower the back‑end concurrency threshold to avoid DB overload while the cache warms.

Scenario 9 – Delete‑First vs. Update‑First Debate

The answer depends on whether the risk is on the read path or the write path. For read‑heavy, write‑light workloads, the safest flow remains:

Update DB.

Delete cache.

If writes are very frequent and stale reads cannot be tolerated, additional measures are needed:

Force reads to hit the DB or perform a forced refresh.

Convert changes into events and let consumers rebuild the cache.

Prefer logical expiration over immediate deletion.

Delayed double‑delete is a patch, not a silver bullet; it only mitigates the "read‑write window" problem and does not solve message loss, ordering, or replication lag.

public void updateProductThenDeleteTwice(ProductDTO product) {
    productMapper.updateById(product);
    String cacheKey = "product:" + product.getId();
    redisTemplate.delete(cacheKey);
    CompletableFuture.runAsync(() -> {
        try { Thread.sleep(300); } catch (InterruptedException e) { Thread.currentThread().interrupt(); }
        redisTemplate.delete(cacheKey);
    });
}

Scenario 10 – Eviction Policy Mis‑configuration

Cache failures often show early warning signs days before a crash: memory approaching limit, eviction count rising, hit rate slowly dropping while response time stays normal.

Typical signs:

Redis memory near max, eviction rate climbing.

Hit rate declines gradually, but latency unchanged.

Hot data gets evicted, causing a surge of DB reads.

Configuration must answer:

Is the instance pure cache or also a lock/queue?

Does eviction affect all keys or only those with TTL?

Should different business domains be isolated into separate instances?

maxmemory 8gb
maxmemory-policy volatile-lru

"volatile‑lru" only evicts keys with an expiration time. If you store permanent configuration data without TTL, they will never be evicted, potentially starving memory for hot data.

Scenario 11 – Master‑Slave Failover Data Loss

After enabling Sentinel or cluster, many assume "if the master dies, nothing breaks". Redis replication is asynchronous, so a failover window can lose data.

Can the business tolerate a few seconds of data loss?

Is the lost data cache only, lock state, idempotency flag, or inventory reservation?

Will clients retry and amplify traffic during the switch?

Persistence settings are a baseline, not a full solution:

appendonly yes
appendfsync everysec
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

Critical state (inventory, payment idempotency, order status) must always be persisted in DB or a reliable log; Redis should only accelerate access.

Scenario 12 – Degrade & Circuit‑break on Cache Failure

When the cache completely fails, the system must re‑allocate resources. The key question is not "can we return the full response" but "which capabilities must survive".

Core fields for a product detail page: title, price, stock status, order entry.

Non‑core data (comments, recommendations, rich media) can be degraded.

@SentinelResource(value = "queryProduct", fallback = "queryProductFallback")
public ProductDTO queryProduct(Long productId) {
    return loadProduct(productId);
}

public ProductDTO queryProductFallback(Long productId, Throwable t) {
    return ProductDTO.degraded(productId);
}

Effective degradation requires a source for fallback data: static templates, recent local cache, or a graceful UI that can render partial information.

Scenario 13 – Missing Monitoring (average metrics hide problems)

Cache health is more than "Redis is up". At minimum monitor:

Hit rate – does the cache actually offload the DB?

Back‑source QPS – indicates DB pressure.

Per‑shard CPU / network – reveals hotspot slots.

Keyspace hit/miss – validates business‑level hit rate.

Eviction count – early sign of capacity issues.

Slow‑query count – catches BigKey or blocking commands.

Client connections – detects leaks or retry storms.

Replication lag – assesses failover risk.

Alert thresholds must consider combinations, e.g., "hit‑rate drop + back‑source QPS rise + DB latency increase" is a true alarm, whereas any single metric alone can be misleading.

Scenario 14 – Redis on Kubernetes

Running Redis in K8s does not automatically make it cloud‑native; it exposes many state‑related pitfalls:

Pod recreation changes network identity; clients may not notice quickly.

Local disk loss slows replica recovery.

Mis‑configured readiness probes let traffic reach unhealthy pods.

Applications hard‑coded to a fixed IP bypass service discovery.

Basic K8s best‑practices:

Use a StatefulSet to keep stable identities.

Expose a headless service or an operator for discovery.

Persist data on a volume; never store production data in the container layer.

Connect via service name, not pod IP.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis
spec:
  serviceName: redis-headless
  replicas: 3
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:7.2-alpine
        args: ["--appendonly", "yes"]
        ports:
        - containerPort: 6379
        volumeMounts:
        - name: redis-data
          mountPath: /data
  volumeClaimTemplates:
  - metadata:
      name: redis-data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 10Gi

For most teams, a managed Redis service from a cloud provider reduces operational burden and eliminates many of these pitfalls.

Why Teams Still Fall After Knowing the Pitfalls

Cache design difficulty lies not in the concepts but in decision boundaries. The same solution can be perfect for one business and disastrous for another. The following decision matrix shows realistic trade‑offs rather than "best practice" recommendations.

Low QPS, no obvious hot spots → single Redis + Cache‑Aside (simple, low maintenance).

Clear hot‑read traffic → add Caffeine local cache (reduces single‑shard pressure).

Allow second‑level stale data → logical expiration + async refresh (stable throughput).

Write‑heavy, read‑many, eventual consistency acceptable → DB‑update then delete‑cache + MQ compensation.

Strong state correctness required → treat Redis as acceleration only; persist state in DB.

Inexperienced K8s team → use managed Redis to avoid operational complexity.

Each upgrade exchanges one cost for another: multi‑level cache adds staleness management; distributed locks limit throughput; eventual consistency adds debugging complexity; self‑built clusters increase ops load.

Common Misconceptions

High hit rate = success – it may only reflect long‑tail data, not real hot paths.

Redis HA = business data safety – asynchronous replication does not guarantee strong consistency.

Local cache always speeds up – with frequent writes or many instances it can create inconsistency windows.

Delayed double‑delete solves consistency – it only mitigates a narrow race condition, not message loss or ordering issues.

Cache clear will self‑recover – without back‑source throttling or degradation, the recovery phase can crash the DB again.

Pre‑deployment Checklist (12 Items)

Define a unified key naming, TTL, and null‑value policy.

Identify which data may return stale values and which must be fresh.

Set mutual‑exclusion or async refresh for hot‑key rebuilds.

Limit maximum concurrent back‑source traffic after cache expiry.

Detect and split BigKey objects.

Configure alerts for eviction count, back‑source QPS, shard CPU, and slow queries.

Test Redis failover, restart, and short‑term unavailability scenarios.

Validate cache cold‑start pressure on the DB after a release.

Ensure critical state lives in DB or a reliable log, not only in Redis.

Check local cache capacity, TTL, and impact during scaling.

Define retry and fallback mechanisms for MQ compensation or Binlog subscription.

Run a full‑scale failure drill, not just a documented plan.

Final Verdict

The biggest trap of distributed caching is not technical complexity but the illusion of simplicity during normal operation, which leads teams to underestimate its cost in failure, scaling, release, and consistency scenarios.

For Spring Boot teams, the optimal cache architecture is never the most complex one; it is the simplest design that:

Clearly knows where it can fail.

Has a well‑defined fallback for each failure mode.

Knows when to stay simple and when to evolve to a richer solution.

When those three questions are answered, Redis becomes a true buffering layer rather than the starting point of the next incident.