Distributed Rate Limiting with Redis + Lua: Surviving API Floods in Spring Boot

Problem: API Flood Overwhelms Database

A GET /api/products/search endpoint normally handled low QPS, but an external integration began sending massive repeated requests. Logs showed the same endpoint called continuously. Application CPU was fine, but the database connection pool saturated, causing cascading failures: rising response times, exhausted connections, other endpoints slowing, gateway timeouts, and client retries amplifying traffic.

Why Local Rate Limiting Fails in Clusters

The initial idea was a simple in-memory counter ( AtomicInteger) allowing 100 requests/second. However, the deployment runs four Spring Boot instances behind Nginx. Each instance maintains its own counter, so a cluster-wide limit of 100 becomes 400 (100 per instance). Requests from the same user may land on different instances, breaking per-user limits. Local solutions ( AtomicInteger, ConcurrentHashMap, Guava RateLimiter, Caffeine) only protect a single JVM.

Distributed Fixed-Window Counter with Redis + Lua

Move the counter to Redis so all instances share one key per user (e.g., rate:search:user:10001). A naive Redis implementation uses separate INCR and EXPIRE commands, risking a key without TTL if the application crashes between them, permanently blocking the user. A GET -then- INCR sequence also introduces race conditions under concurrency.

Solution: execute the entire logic atomically in a Lua script on the Redis server.

local key = KEYS[1]
local limit = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local current = redis.call('INCR', key)
local ttl = redis.call('PTTL', key)
if current == 1 or ttl < 0 then
  redis.call('PEXPIRE', key, window)
  ttl = window
end
if current > limit then
  return {0, current, ttl}
end
return {1, current, ttl}

The script increments the counter, sets the expiry (in milliseconds) only on the first request or if the key has no TTL, then checks the limit. It returns a tuple: allowed (1/0), current count, remaining TTL.

Spring Boot Integration

Register the script as a DefaultRedisScript bean loading from scripts/rate_limit.lua. Create a RedisRateLimiter component that executes the script via StringRedisTemplate, passing the key, limit, and window in milliseconds. It returns a RateLimitResult record ( allowed, current, retryAfterMillis).

Annotation-Driven AOP Aspect

Define a @RateLimit annotation with attributes: limit, windowSeconds, dimension (USER, IP, API_KEY, GLOBAL), and optional key for custom resource names. An aspect ( RateLimitAspect) intercepts annotated methods, resolves the identity via a RateLimitKeyResolver, builds the Redis key ( rate:{resource}:{identity}), calls the limiter, and throws a custom RateLimitExceededException with the retry-after value if denied.

Identity Resolution & Security Considerations

USER : extracts authenticated username from Spring Security context; falls back to "anonymous".

API_KEY : reads X-API-Key header, hashes it with SHA-256 before using as Redis key part to avoid storing raw credentials.

IP : uses request.getRemoteAddr(). Warns that behind CDN/Nginx/Ingress this returns the proxy IP. Blindly trusting X-Forwarded-For allows attackers to forge IPs and bypass limits. Correct approach: trust only headers set by your own controlled reverse proxies, and strip client-supplied values at the gateway.

GLOBAL : uses constant "global" for cluster-wide limits.

Proper HTTP 429 Response

Instead of throwing a generic 500, the exception handler returns 429 Too Many Requests with a Retry-After header (seconds) and a JSON body containing code, message, and retryAfter. This tells clients the service isn't broken, just rate-limited, and when to retry — preventing aggressive immediate retries that worsen the flood.

Fixed Window Limitations & Algorithm Choice

The implementation uses a fixed time window (e.g., 60 seconds). At window boundaries, a burst of 2× the limit can occur (100 requests at second 59, another 100 at second 61). Fixed window is acceptable for:

General API anti-scraping

Login attempt limits

Export endpoint protection

Low-cost quota enforcement

For high-cost operations (LLM APIs, SMS, paid third-party calls, expensive DB queries), prefer Token Bucket or Sliding Window for smoother burst control. The author advocates starting simple and upgrading only when boundary bursts become a real problem.

Handling Redis Failures: Fail-Open vs Fail-Closed

No single answer. For non-critical endpoints (e.g., product search), fail-open : log error, allow request, alert ops. For critical/high-cost endpoints (SMS, login brute-force protection, expensive AI APIs), fail-closed or degrade to a local RateLimiter as a temporary shield. The strategy should be configurable per endpoint, not hardcoded, because the rate limiter itself must not become a single point of failure that takes down the whole business.

Observability

Emit metrics: rate_limit_allowed_total, rate_limit_rejected_total, rate_limit_redis_error_total, dimensioned by endpoint, user type, tenant. Monitoring reveals misconfigured limits (too loose → ineffective; too tight → false positives) before users complain.

Key Takeaway

Rate limiting isn't just for attackers. Production incidents often stem from buggy retry loops (no backoff), misconfigured cron jobs, sudden partner traffic spikes, or new version rollouts. All generate legitimate but excessive requests. When designing a new endpoint, ask: "If 10,000 requests hit this simultaneously, what should happen?" If the answer is "let them all in", you're delegating the decision to Tomcat thread pools, connection pools, and MySQL — which randomly decide who fails first. Rate limiting puts that control back in your hands.