Why Add Nginx Rate Limiting When Spring Cloud Gateway Already Provides It?

How Gateway Rate Limiting Works

Spring Cloud Gateway includes RequestRateLimiterGatewayFilterFactory, which uses Redis + a Lua script to implement a token‑bucket algorithm. A minimal configuration looks like:

spring:
  cloud:
    gateway:
      routes:
        - id: order-service
          uri: lb://order-service
          filters:
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 500   # generate 500 tokens per second
                redis-rate-limiter.burstCapacity: 1000 # bucket size
                key-resolver: "#{@userKeyResolver}"   # limit per user

For each request, the gateway executes the Lua script in Redis to deduct a token; if a token is available the request passes, otherwise a 429 response is returned. This works only if the request first reaches the gateway process.

Gateway Can Be Overwhelmed

Running on the JVM with Netty, a single gateway instance on a 4‑core 8 GB machine handles roughly 20 k–30 k QPS, depending on filter complexity and downstream latency. If the configured limit is 5 k requests per second, normal traffic is throttled correctly, but a burst of 100 k QPS (e.g., during a promotion) forces every request through the gateway.

Accept TCP connection and complete TLS handshake

Parse HTTP headers and body

Execute filter chain (authentication, logging, routing)

Call Redis to run the token‑bucket Lua script

Construct a 429 response and write it back

Even if the rate‑limit check takes only ~2 ms, 100 k concurrent requests exhaust the Netty thread pool, trigger frequent Young GC pauses, and can lead to connection‑queue overflow, timeouts, or OOM. In short, the gateway crashes before the limit logic runs.

Why Nginx Rate Limiting Is Powerful

Nginx, written in C with an event‑driven multi‑process model, can easily handle >100 k concurrent connections with microsecond‑level per‑request overhead. Its ngx_http_limit_req_module uses a leaky‑bucket algorithm and is configured as follows:

http {
    # Define a limit of 5 000 requests per second per client IP, shared memory 10 MB
    limit_req_zone $binary_remote_addr zone=api_limit:10m rate=5000r/s;

    server {
        location /api/ {
            limit_req zone=api_limit burst=2000 nodelay; # allow burst of 2000, excess returns 503 immediately
            proxy_pass http://gateway_upstream;
        }
    }
}

When 100 k QPS arrives, Nginx performs the limit check entirely in‑process (a simple shared‑memory counter), returning 503 for 95 % of requests without ever reaching the gateway. The gateway only processes the allowed 5 k QPS, well within its capacity.

Different Responsibilities of Two Layers

Nginx layer: coarse‑grained, global traffic control based on IP, path, or overall QPS. It acts as a physical firewall, blocking excess traffic before it reaches downstream services.

Gateway layer: fine‑grained, business‑level control such as per‑user, per‑merchant, or per‑API‑parameter limits. Example of a per‑user limit:

@Bean
public KeyResolver userKeyResolver() {
    return exchange -> Mono.just(
        exchange.getRequest().getHeaders().getFirst("X-User-Id")
    );
}

This requires inspecting request headers or tokens, which can only be done at the application layer.

Typical Production Architecture

In medium‑sized companies, rate limiting often spans three or four layers, each progressively closer to the client and each with decreasing processing capacity but increasing granularity. The diagram below illustrates such a stack (image omitted for brevity).

Placing all limits only at the gateway would concentrate all traffic pressure on a single JVM process, similar to having a door‑access control only at the building entrance while the elevator shaft remains unprotected.

Gateway Restarts and GC Impact

During a rolling deployment, some gateway instances are offline, increasing load on the remaining ones and potentially requiring dynamic adjustment of limits. Full GC pauses can last hundreds of milliseconds to seconds, during which the gateway cannot handle requests, effectively disabling its rate‑limit logic. Nginx limits remain unaffected because they operate outside the JVM.

Conclusion

Spring Cloud Gateway’s rate limiting handles business‑level, fine‑grained traffic control, while Nginx provides infrastructure‑level protection that shields the gateway from overload. The core principle is to stop traffic upstream before it can overwhelm downstream components, and Nginx fulfills that role.