AI-Generated Cache & Rate Limiting Code: Hidden Concurrency Bugs Exposed by Stress Testing

Failure Scene: Stress Test Triggers Database CPU Spike

A property listing query API was protected by AI-generated Redis caching and rate limiting. The code included Bloom filter for cache penetration, mutex lock for cache breakdown, random TTL for cache avalanche, and a Redis counter for rate limiting. Local single-threaded tests passed, but a 200-concurrency stress test caused MySQL CPU to hit 95%, connection exhaustion, and request timeouts. Monitoring showed massive queries for non-existent keys and the rate limiter failing to block bursts.

Bug 1: Bloom Filter Only Initialized Once at Startup

AI approach: Redisson Bloom filter checks key existence before cache lookup; missing keys return null immediately.

// AI-generated snippet
public HouseDTO getHouseInfo(Long houseId) {
    if (!bloomFilter.contains(houseId)) {
        return null;
    }
    String cacheKey = "house:info:" + houseId;
    String cacheData = redisTemplate.opsForValue().get(cacheKey);
    if (cacheData != null) {
        return JSON.parseObject(cacheData, HouseDTO.class);
    }
    HouseDO houseDO = houseMapper.selectById(houseId);
    if (houseDO == null) {
        return null;
    }
    redisTemplate.opsForValue().set(cacheKey, JSON.toJSONString(convert(houseDO)), 30, TimeUnit.MINUTES);
    return convert(houseDO);
}

Defect: The Bloom filter is populated only once at application startup. New property listings are never added to it, so legitimate new IDs are rejected as non-existent. The AI omitted synchronization logic for data inserts/updates.

Many AI-generated Bloom filter implementations assume a static dataset and ignore continuous business data growth.

Bug 2: Mutex Lock Uses Recursive Retry Causing Stack Overflow

AI approach: On cache miss, acquire a Redis distributed lock via setIfAbsent; the lock holder queries DB and repopulates cache, while others sleep 100 ms and recurse.

// AI mutex lock pseudocode
if (cacheData == null) {
    boolean lock = redisTemplate.opsForValue().setIfAbsent("lock:house:" + houseId, "1", 10, TimeUnit.SECONDS);
    if (lock) {
        // query DB, write cache
    } else {
        Thread.sleep(100);
        return getHouseInfo(houseId); // recursive retry
    }
}

Defect: Recursive retries under high concurrency rapidly accumulate stack frames, causing StackOverflowError. No maximum retry limit means threads block indefinitely. If the lock holder crashes, the lock expires after 10 seconds while thousands of threads spin, exhausting the servlet thread pool.

Bug 3: Random TTL Range Too Narrow; Hot Keys Not Handled

AI approach: Base TTL 30 minutes plus new Random().nextInt(300) seconds (0–5 min) to stagger expiration.

// AI code
long expire = 30 * 60 + new Random().nextInt(300);
redisTemplate.opsForValue().set(cacheKey, data, expire, TimeUnit.SECONDS);

Defects: (1) new Random() inside a hot loop creates excessive objects, degrading throughput. (2) The 5-minute jitter is insufficient for hot keys — popular listings still risk simultaneous expiration. The AI applied a generic template without distinguishing hot keys, which require no expiration + background async refresh .

Bug 4: Rate Limiter Uses Non-atomic INCR + EXPIRE

AI approach: Per-IP counter with 1-second window, 10 requests/second limit.

// AI rate limiting code
String limitKey = "limit:ip:" + ip;
Long count = redisTemplate.opsForValue().increment(limitKey, 1);
if (count == 1) {
    redisTemplate.expire(limitKey, 1, TimeUnit.SECONDS);
}
if (count > 10) {
    throw new RateLimitException("请求过于频繁");
}

Defect: INCR and EXPIRE are two separate Redis commands. Under concurrency, thread A increments but hasn't set expiry; thread B increments again. The key may never receive an expiry, causing the counter to grow unbounded and permanently block the IP.

AI often treats two independent commands as atomic, forgetting to wrap them in a Lua script — a classic interview question that still slips into generated business code.

Fixes Applied

1. Bloom Filter Synchronization

On new listing creation or deletion, call bloomFilter.add(houseId) synchronously. A scheduled job incrementally syncs database IDs to the filter as a safety net against inconsistency.

2. Mutex Lock Rewrite

Replace recursion with bounded spin-wait (max retry count). Adopt Redisson distributed lock, which provides automatic lease renewal (watchdog) to prevent premature release during long DB operations.

3. Cache Avalanche Mitigation

Normal keys: base TTL + random offset (use ThreadLocalRandom for performance).

Hot keys: set no TTL; a dedicated background scheduler asynchronously refreshes their values, eliminating breakdown risk at the source.

4. Atomic Rate Limiting via Lua Script

-- Rate limiting Lua script
local key = KEYS[1]
local limit = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local current = redis.call('INCRBY', key, 1)
if current == 1 then
    redis.call('EXPIRE', key, window)
end
return current <= limit

Java invokes the script via redisTemplate.execute(); the entire check-and-increment runs atomically inside Redis, eliminating the race condition.

Stress Test Verification (200 Concurrent)

Non-existent IDs intercepted by Bloom filter — zero DB penetration.

Hot key expiration triggers only a handful of DB calls.

Rate limiter counters reset cleanly; no permanent blocking.

MySQL CPU stable below 30%, zero timeouts.

Checklist: Four Must-Check Points for AI-Generated Cache/Rate-Limit Code

Bloom filter: Static initialization ≠ real-time sync; every data insert must update the filter.

Distributed/mutex lock: Ban recursive retries; enforce max retries; prefer battle-tested Redisson over hand-rolled locks.

Cache avalanche: Random TTL only mitigates; hot keys need dedicated async refresh.

Redis rate limiting: INCR + EXPIRE must be atomic — always use a Lua script; two separate commands are a guaranteed landmine.

Remember: AI-produced caching, rate limiting, and distributed-system code must not go live without concurrency stress testing . Unit tests never reveal these timing bugs.