Why Spring Cloud Gateway Rate Limiting Fails to Protect Downstream Services

01 Gateway Rate Limiting Configuration

The typical Spring Cloud Gateway setup uses RequestRateLimiter with spring-boot-starter-data-redis-reactive. A route configuration example:

spring:
  cloud:
    gateway:
      routes:
      - id: order_service_route
        uri: lb://order-service
        predicates:
        - Path=/order/**
        filters:
        - name: RequestRateLimiter
          args:
            key-resolver: "#{@apiKeyResolver}"
            redis-rate-limiter.replenishRate: 100
            redis-rate-limiter.burstCapacity: 200
            redis-rate-limiter.requestedTokens: 1

A KeyResolver extracts user ID from the X-User-Id header, defaulting to anonymous:

@Bean
public KeyResolver apiKeyResolver() {
    return exchange -> Mono.justOrEmpty(exchange.getRequest().getHeaders().getFirst("X-User-Id"))
            .defaultIfEmpty("anonymous");
}

Parameters: replenishRate: 100 (tokens/second, steady QPS limit), burstCapacity: 200 (bucket capacity for absorbing spikes), requestedTokens: 1 (tokens per request). Low-concurrency tests pass: exceeding 100 QPS returns 429. Yet burst traffic still crashes downstream.

02 Reason One: QPS Limiting Controls Entry Rate, Not In-Flight Concurrency

Many assume limiting entry QPS to 100 means downstream sees 100 QPS load. This is wrong. Little's Law governs the relationship:

In-flight Concurrency = QPS × Average Response Time (RT)

Normal case: order API RT = 20ms. At 100 QPS, concurrency = 100 × 0.02 = 2 concurrent requests. Tomcat default 200 threads handles this easily.

Burst scenario: resource contention (DB row locks, Redis hot keys) degrades RT from 20ms to 2000ms. Gateway still admits 100 requests/second, but each request now occupies a thread for 2 seconds. In-flight requests accumulate: 100 QPS × 2s = 200 concurrent requests. Thread pool saturates, DB connection pool exhausts, cascade failure occurs. The gateway limits entry rate but cannot sense downstream in-flight count.

03 Reason Two: burstCapacity Generates Pulse Traffic

burstCapacity: 200

allows the bucket to accumulate 200 tokens during idle periods. At flash-sale start (00:00:00.000), 200 requests arrive within 1ms, all acquire tokens, and gateway releases them in 1ms. Downstream receives 200 concurrent requests instantly instead of 100 spread over 1 second. With a typical DB pool of 10-20 connections, the pool exhausts immediately; all threads block waiting for connections. burstCapacity intends to absorb jitter but becomes an uncontrollable pulse for DB-backed services with variable latency.

04 Reason Three: Multiple Routes Multiply Quotas

Production gateways run 2-4 instances. RedisRateLimiter uses shared Redis + Lua, so cluster-wide rate holds. However, teams often split routes (e.g., order_normal_route for /order/detail/** and order_pay_route for /order/pay/**), each with replenishRate: 100. The Redis key includes route ID: request_rate_limiter.{routeId}.{apiKey}. Thus two independent token buckets exist. Simultaneous traffic on both routes yields 200 QPS into the same order-service, doubling the configured limit while each route reports "within limit".

Worse, some teams replace Redis with local limiters (Guava RateLimiter, Bucket4j) to reduce RTT. With 5 gateway instances each configured at 100 QPS, actual downstream throughput becomes 500 QPS — 5× the intended limit.

05 Reason Four: KeyResolver Hidden Flaws

The KeyResolver fallback to anonymous merges all unauthenticated/crawler/malicious traffic into one bucket. A flood of header-less requests consumes the 100 tokens, causing legitimate unauthenticated users to receive 429.

Second flaw: if KeyResolver returns Mono.empty() (no defaultIfEmpty), the denyEmptyKey parameter controls behavior. If misconfigured as false:

spring:
  cloud:
    gateway:
      filter:
        request-rate-limiter:
          deny-empty-key: false  # misconfigured

Requests with unresolvable keys skip the rate limiter entirely and pass through. During a burst of unauthenticated requests, the rate limit becomes ineffective.

06 Reason Five: Redis Becomes the Bottleneck

Gateway (Netty + Reactor) handles high throughput. Each request triggers a Lua script via Lettuce:

-- request_rate_limiter.lua core fragment
local rate = tonumber(ARGV[1])
local capacity = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local requested = tonumber(ARGV[4])
local fill_time = capacity / rate
local ttl = math.floor(fill_time * 2)
local last_tokens = tonumber(redis.call("get", tokens_key))
-- calculate refill and deduct...

Under tens of thousands QPS burst, Redis single-threaded CPU saturates executing Lua scripts. Latency jumps from 0.5ms to 20ms+. Netty EventLoop backs up,

RedisCommandTimeoutException</sub> surges. The rate limiter's fault-tolerance logic then fails open:</p><pre><code>[WARN] Error determining whether user is allowed: Redis command timed out
-> fail-open, request forwarded downstream!

At the moment protection is most needed, the limiter collapses and the floodgate opens.

07 How to Truly Protect Downstream

Tighten burstCapacity

For DB-dependent core APIs, keep burst capacity close to replenish rate:

redis-rate-limiter.replenishRate: 100
redis-rate-limiter.burstCapacity: 110  # only ~10% headroom
redis-rate-limiter.requestedTokens: 1

Prefer client retries over admitting multiples of pulse traffic.

Gateway Shapes Rate; Downstream Enforces Concurrency Isolation

Add Sentinel or Resilience4j at the service level, limiting by concurrent threads not QPS:

@SentinelResource(value = "createOrder", blockHandler = "handleBlock")
public OrderVO createOrder(OrderDTO dto) {
    // Even if gateway admits 200 requests, downstream runs max 20 threads concurrently
    // Excess fails fast, never exhausts DB connection pool
    return orderService.doCreate(dto);
}

public OrderVO handleBlock(OrderDTO dto, BlockException ex) {
    throw new BusinessException("System busy, try later");
}

Sentinel rule (grade 0 = thread count):

[
  {
    "resource": "createOrder",
    "grade": 0,
    "count": 20  // max 20 concurrent threads
  }
]

With concurrency isolation, even if RT degrades from 20ms to 2s, only 20 threads are occupied; the remaining 180 Tomcat threads serve other endpoints, preventing total collapse.

Harden KeyResolver

Set denyEmptyKey: true and return 401 for missing keys:

spring:
  cloud:
    gateway:
      filter:
        request-rate-limiter:
          deny-empty-key: true
          empty-key-status-code: 401

Use a composite resolver: authenticated users by user ID, unauthenticated by client IP:

@Bean
public KeyResolver compositeKeyResolver() {
    return exchange -> {
        String userId = exchange.getRequest().getHeaders().getFirst("X-User-Id");
        if (StringUtils.hasText(userId)) {
            return Mono.just("user_" + userId);
        }
        String clientIp = exchange.getRequest().getRemoteAddress().getAddress().getHostAddress();
        return Mono.just("ip_" + clientIp);
    };
}

08 Closing Thoughts

Configuring a gateway RateLimiter does not solve high-concurrency problems. True resilience requires layered defense: gateway shapes entry rate, downstream enforces concurrency isolation and circuit breaking, storage layer protects connection pools. Each layer guards its own boundary. Gateway rate limiting protects the entrance; downstream concurrency isolation protects the service internals. Both are indispensable.