Spring Cloud Gateway Internals: Reactive Model, Filter Chains & High-Concurrency Tuning

Nacos solves "where is the service", Feign solves "how to call", and this article solves "who guards the gate". The gateway is the sole external entry point of a microservice system and the most critical hop that must not fail: it is the most memory-sensitive, handles the highest concurrency, and any blocking call amplifies into a site-wide outage. Spring Cloud Gateway has only three abstractions—Route, Predicate, Filter—but truly mastering it requires understanding WebFlux's non-blocking model, the filter chain's pre/post reverse-order mechanism, and the four-layer configuration of connection pools and timeouts. This article breaks down the topic into four parts: incidents, model, source code, and tuning.

1. Three Real Incidents

Incident 1: Gateway CPU at 100%, full-site timeout. Investigation revealed a custom filter that logged by joining the request body into a string and performed a synchronous Redis query. Netty's EventLoop threads default to twice the CPU cores; one blocked thread caused thousands of connections on that thread to queue up. The gateway showed 100% CPU, but threads were actually stuck.

Incident 2: Routes correct, but persistent 404. Configuration used Path=/api/order/** with StripPrefix=1, intending to strip the api prefix before forwarding to order-service. However, the downstream controller mapped to /order/list; stripping one layer left /order/list, which looked correct—but the frontend actually requested /api/order/order/list (an extra layer), so stripping still produced a wrong path. Prefix stripping guesses the contract rather than matching it.

Incident 3: Rate limiting configured, but flash sale still broke through. The RequestRateLimiter 's keyResolver returned a constant "all", causing the entire site to share one token bucket. Additionally, replenishRate was misunderstood as "tokens per minute" when it is actually per second . Two misconceptions combined rendered rate limiting ineffective.

These three incidents correspond to three core issues: blocking forbidden zone (incident 1), routing contract (incident 2), rate limiting semantics (incident 3). They are not about "how to configure a route" but "how the gateway withstands full-site traffic".

2. Technical Analysis: Why the Gateway Must Be Reactive

2.1 What the Gateway Actually Does

Stripping away component names, the gateway has only four responsibilities: Routing (map URL to service instance), Traffic Governance (rate limiting, circuit breaking, retry, canary), Security Boundary (auth, anti-replay, desensitization), Protocol Adaptation (HTTP to WebSocket, internal/external protocol conversion). The first two determine performance; the last two determine whether it can exist as a unified entry point.

2.2 Selection: Boundaries of Five Entry Points

Nginx / OpenResty: King of static resources and Layer 4 proxy; dynamic logic requires Lua, disconnected from Java ecosystem, service discovery needs manual upstream maintenance.

APISIX / Kong: Cloud-native gateways based on OpenResty, strong plugin ecosystem, excellent performance, suitable as traffic entry point (north-south), but business-side custom logic requires Lua or plugin system.

Zuul 1: Servlet blocking model, one thread per request, no longer evolving, avoid for new projects.

Spring Cloud Gateway: WebFlux + Reactor Netty, seamless with Spring ecosystem (reuses Security, Micrometer, Resilience4j), suitable as unified entry for business microservices , lowest marginal cost for Java teams.

Common production combination: outer layer uses APISIX/Nginx for Layer 4 and WAF; inner layer uses Gateway for business routing and auth. They are not competitors—outer layer solves "can traffic enter", inner layer solves "where to go after entry, who has permission, is limit exceeded".

2.3 Why WebFlux Is Mandatory

In the Servlet blocking model, one request occupies one thread. 200 worker threads support only 200 concurrent requests . As the full-site entry point, the gateway handles tens of thousands of connections; the thread model is a death sentence—adding threads only amplifies context-switching and memory overhead.

WebFlux runs on Reactor Netty. EventLoop count defaults to CPU cores × 2 (4 cores = 8 threads), handling connections via callbacks and event-driven processing: while waiting for downstream response, the thread immediately serves other connections. Tens of thousands of long connections need only dozens of threads. The cost: zero blocking allowed anywhere in the chain .

// Blocking: 10k connections -> 10k threads
// 8MB stack per thread, total 80GB
// Reactive: 10k connections -> 8 EventLoops
// Each connection uses only small off-heap buffers
// Cost: ANY blocking on the chain
// drags down ALL connections on the same EventLoop

This explains why ThreadLocal fails , MDC doesn't propagate , transactional annotations are meaningless in the gateway—they all assume "one request, one thread, synchronous execution within thread", a premise broken in the reactive model. Use Reactor Context for context propagation, not ThreadLocal.

2.4 Three Core Abstractions

Route: Minimal forwarding unit, composed of id, target URI, a set of predicates, a set of filters, metadata. ID must be unique; URI supports http, lb (load balancing), ws, forward protocols.

Predicate: Decides "does this request match this route", essentially Predicate<ServerWebExchange>. Eleven built-ins: Path, Method, Header, Query, Cookie, Host, RemoteAddr, Weight, After, Before, Between. Multiple predicates are AND relationship; all must be true for a match.

Filter: Processing unit on the responsibility chain, divided into Route-level (GatewayFilter, configured in routes.filters, effective only for that route) and Global-level (GlobalFilter, effective for all routes). Both are merged into one chain, sorted by order.

2.5 Minimal Working Configuration

spring:
  cloud:
    gateway:
      httpclient:
        connect-timeout: 1000
        response-timeout: 3s
      routes:
        - id: order
          uri: lb://order-service
          predicates:
            - Path=/api/order/**
          filters:
            - StripPrefix=1
            - name: RequestRateLimiter
              args:
                replenishRate: 100
                burstCapacity: 200
                key-resolver: "#{@ipKey}"

Two easily overlooked points: connect-timeout and response-timeout under httpclient are global , not per-route; per-route timeouts use metadata connect-timeout / response-timeout with higher priority.

3. Core Flow: Six Stages of a Request

Understanding the gateway is best done by tracing a request end-to-end. Six stages correspond to six observable points; troubleshoot by stage.

① Netty receives HTTP message
    HttpServerOperations -> ReactorHttpHandlerAdapter
② WebFlux dispatches request
    DispatcherHandler -> RoutePredicateHandlerMapping
③ Route matching (traverse all Routes)
    All predicates true -> selected, write GATEWAY_ROUTE_ATTR
④ Filter chain assembly
    Route-level + Global -> sort -> FilteringWebHandler
⑤ Chain execution (pre forward / post reverse)
    LoadBalancerClientFilter replaces lb:// with real IP
    NettyRoutingFilter actually sends request
⑥ Response write-back
    NettyWriteResponseFilter(-1) writes response to client

3.1 Route Matching: First Match Wins

RoutePredicateHandlerMapping

traverses all routes in configuration order , evaluating predicates. Once a route's predicates are all true, it stops immediately and puts the Route into exchange attributes—classic "first match wins", so route order is priority ; broad routes must be placed later.

3.2 How lb:// Becomes Real IP

The configured lb://order-service is not a valid URL. Replacement happens in ReactiveLoadBalancerClientFilter (order 10150) : it extracts serviceId from exchange, asks LoadBalancer to pick an instance, rewrites the URL to http://ip:port/..., writes to GATEWAY_REQUEST_URL_ATTR for subsequent filters. This is why the gateway must include LoadBalancer—it not only "picks a machine" but also performs URL reconstruction .

3.3 Two-Level Filter Chain Ordering

Chain order is determined by order value, smaller first. Execution enters pre-phase in ascending order, reaches chain tail, then post-phase executes in reverse order —this "onion model" is the core of all Filter design.

4. Core Source Code: Five Must-Read Sections

4.1 RoutePredicateHandlerMapping.lookupRoute: Match and Win

protected Mono<Route> lookupRoute(
    ServerWebExchange ex) {
    return this.routeLocator.getRoutes()
    // Keep if predicate true, else filter out
    .filterWhen(r -> r.getPredicate()
    .apply(ex))
    .next() // Take first, short-circuit immediately
    .switchIfEmpty(Mono.error(
    new ResponseStatusException(
    HttpStatus.NOT_FOUND)));
}

Note the next() operator: it cancels upstream subscription after first element, so subsequent routes are never evaluated. Direct engineering conclusion: route configuration order equals priority ; placing /api/order/** after /api/** means order requests never reach their route.

4.2 FilteringWebHandler.handle: Merging Two Filter Levels

public Mono<Void> handle(
    ServerWebExchange ex) {
    Route route = ex.getRequiredAttribute(
    GATEWAY_ROUTE_ATTR);
    // Route-level first, global after, then unified sort
    List<GatewayFilter> list
    = new ArrayList<>(route.getFilters());
    list.addAll(this.globalFilters);
    AnnotationAwareOrderComparator.sort(list);
    return new DefaultGatewayFilterChain(list)
    .filter(ex);
}

These ten lines are the gateway's scheduling center. Two key points: Route-level and global filters are equal , merged and sorted together—whoever has smaller order runs first, no "global always before route-level"; sorting uses AnnotationAwareOrderComparator , recognizing both Ordered interface and @Order annotation, so custom filters work either way.

4.3 DefaultGatewayFilterChain.filter: Essence of Reverse Order

public Mono<Void> filter(
    ServerWebExchange ex) {
    return Mono.defer(() -> {
    if (this.index < filters.size()) {
    int i = this.index;
    GatewayFilter f = filters.get(i);
    DefaultGatewayFilterChain next =
    new DefaultGatewayFilterChain(
    this, i + 1);
    return f.filter(ex, next);
    }
    return Mono.empty();
    });
}

Chain invocation uses no loop; instead each level creates a new chain object with index+1 , passing "remaining chain" as parameter to current filter. Thus in f.filter(ex, next), code before next.filter() is pre, after is post —because next recurses to tail then returns layer by layer, the return path is naturally reverse order. This is not deliberate framework design but inevitable result of recursion.

// Onion model: pre forward, post reverse
A.pre -> B.pre -> C.pre -> forward downstream
<- C.post
<- B.post
<- A.post

4.4 NettyRoutingFilter: The Actual Forwarding Point

public Mono<Void> filter(ServerWebExchange ex,
    GatewayFilterChain chain) {
    Route r = ex.getAttribute(
    GATEWAY_ROUTE_ATTR);
    URI url = ex.getAttribute(
    GATEWAY_REQUEST_URL_ATTR);
    HttpClient client = getHttpClient(r, ex);
    HttpClientRequest req
    = client.request(method).uri(url);
    // Write request body/headers here, then send
    return req.response().flatMap(res -> {
    ex.getAttributes()
    .put(CLIENT_RESPONSE_ATTR, res);
    return chain.filter(ex);
    });
}

It is one of the filters with largest order (LOWEST_PRECEDENCE), meaning everything before it is pre-logic, everything after (inside flatMap) is post-logic . Also remember: HttpClient is a global singleton , connection pool lives there, so pool parameters are global under httpclient, cannot be set per-route.

4.5 How Configuration Becomes Filter Instances

YAML's name: RequestRateLimiter is not direct reflection but factory pattern: RouteDefinitionRouteLocator finds RequestRateLimiterGatewayFilterFactory by name, binds args into Config object, calls apply(config) to produce filter instance. Class name is XxxGatewayFilterFactory, config writes Xxx —suffix auto-appended, the most common pitfall for custom filters.

5. Built-in Filter Order Map

For tuning and debugging, remembering a few key filter orders is more useful than memorizing all thirty-plus. Listed by ascending value:

RemoveCachedBody MIN clear body cache
AdaptCachedBody MIN+1k cache body
NettyWriteResponse -1 write response
GatewayMetrics 0 collect metrics
RouteToRequestUrl 10k build target URL
ReactiveLBClient 10150 pick instance rewrite URL
NettyRoutingFilter MAX actual forward

Directly usable inferences: To modify URL before request , order < 10000; To modify real target address , order < 10150; To process downstream response , order > MAX position, or in post-phase (recommend order after NettyWriteResponseFilter for response transformation).

Most worry-free custom order values: Auth use -100 (before all routing logic), Logging/metrics use 0 (same level as metrics), Response transformation use Ordered.LOWEST_PRECEDENCE - 1 (right after forwarding). Don't use Integer.MAX_VALUE, collides with forwarding filter.

6. High-Concurrency Tuning: Four Ledgers

6.1 Thread Ledger: EventLoop vs Business Threads

Two thread types in gateway: EventLoop handles I/O and all non-blocking logic on chain; Business thread pool used only when blocking code is unavoidable. Official advice: avoid blocking if possible; must call legacy interfaces (JDBC, sync SDK), use publishOn(Schedulers.boundedElastic()) to switch off, never call directly on EventLoop.

// Wrong: blocking call on EventLoop
return webClient.get().retrieve()
.bodyToMono(String.class)
// This line drags down EventLoop thread
.map(s -> blockingDao.query(s));

// Correct: switch to elastic pool then block
return Mono.fromCallable(
() -> blockingDao.query(key))
.subscribeOn(Schedulers.boundedElastic());

boundedElastic default upper limit is CPU cores × 10 threads, queue capacity 100k. It exists as "safety net", not "use freely"—once massive requests switch to it, you've degraded gateway back to blocking model, just renamed.

6.2 Connection Ledger: Three Pool Parameters

Gateway-to-downstream connection pool managed by Reactor Netty, three parameters determine high-pressure behavior: maxConnections per-instance max connections (default 500), pendingAcquireMaxCount queue limit when no connection available (default 2× max), pendingAcquireTimeout queue timeout (default 45 seconds).

spring:
  cloud:
    gateway:
      httpclient:
        pool:
          max-connections: 1000
          max-idle-time: 30s
          acquire-timeout: 2000 # queue timeout

Most dangerous is default 45-second queue timeout: when downstream slows, connections exhausted, new requests queue 45 seconds before failing, while they keep occupying upstream connections and memory , quickly dragging gateway down. Production should compress to second-level (e.g., 2 seconds) for fast failure.

6.3 Timeout Ledger: Four Layers Must Decrease

Gateway timeout is the first gate of the whole chain, single principle: closer to user, shorter timeout . Reversed causes "downstream already failed, upstream still waiting" resource waste.

Client / Nginx 10s
Gateway response-timeout 3s
Feign / WebClient 2s
Business method 1.5s
DB / Redis 800ms

connect-timeout separate: 1000ms (cross-region can relax)

Often overlooked detail: response-timeout since 4.x is "overall timeout" not "read timeout", covering from request sent to response headers arrival. It must be smaller than caller (frontend or Nginx) timeout, else user sees timeout page while gateway still waits downstream.

6.4 Memory Ledger: Off-Heap Is the Bulk

Netty uses DirectByteBuffer for send/receive, this memory not in heap , only limited by MaxDirectMemorySize (default equals max heap). Common stress-test phenomenon: heap healthy but container OOMKilled—reason here. Explicitly set -XX:MaxDirectMemorySize and leave container headroom.

Another hidden memory killer: request body caching . WebFlux defaults to caching only bodies ≤256KB, larger throws DataBufferLimitException. If business truly needs large files, use streaming forwarding instead of reading entire body into memory—gateway memory scales with concurrency, one 2MB cache × 2000 concurrent = 4GB.

6.5 Rate Limiting Ledger: Redis Token Bucket

RequestRateLimiter

default implementation runs Lua on Redis, implementing token bucket : replenishRate is tokens added per second, burstCapacity is bucket capacity (allowed instantaneous burst). Equal = steady rate; burst > replenish enables burst capability.

@Bean
KeyResolver ipKey() {
    return ex -> Mono.just(
    ex.getRequest().getRemoteAddress()
    .getAddress().getHostAddress());
}

# Better: rate limit by user + interface
# key = userId + ":" + path

Granularity determines effectiveness: IP-based misinjures entire building in NAT; constant-key equals global rate limiting. Production typically uses "user ID preferred, fallback to IP" composite key, with separate thresholds for login, order placement. Return 429 with Retry-After header when limited.

7. Custom Filters & Auth Practice

Two most common global filters in real projects: auth and distributed logging. Below auth example covers three critical actions— verify, short-circuit, propagate user info .

@Component
public class AuthGlobalFilter
implements GlobalFilter, Ordered {
    @Override
    public Mono<Void> filter(
    ServerWebExchange ex,
    GatewayFilterChain chain) {
    String token = ex.getRequest()
    .getHeaders()
    .getFirst("Authorization");
    if (token == null || !verify(token)) {
    ex.getResponse().setStatusCode(
    HttpStatus.UNAUTHORIZED);
    return ex.getResponse()
    .setComplete();
    }
    // Propagate parsed user info downstream
    ServerHttpRequest req = ex
    .getRequest().mutate()
    .header("X-User-Id", uid(token))
    .build();
    return chain.filter(
    ex.mutate().request(req).build());
    }
    @Override
    public int getOrder() { return -100; }
}

Three must-know points: Short-circuit must return setComplete() not null, else chain continues; Modify request via mutate() , ServerHttpRequest is immutable; Propagate via header not ThreadLocal , latter unreliable in reactive model.

Also, if verify(token) involves remote call (e.g., JWT blacklist check), must use non-blocking client with local cache, else gateway becomes serial auth center, full-site bottleneck. Common pattern: JWT local signature verification + blacklist short-TTL cache, reducing remote calls to near zero.

8. Eight High-Frequency Pitfalls

Blocking code in filter. JDBC, sync Redis client, Thread.sleep, file I/O—any blocks EventLoop. Symptom: CPU not high but all requests stuck, thread stacks full of reactor-http-nio.

Assuming global filter always runs first. Both levels merged and sorted by order; global not inherently prior. Auth filter must explicitly use negative order.

Route order reversed. First match wins; /api/** before children kills all sub-routes. Broad routes must be last.

Custom filter name missing factory suffix. Class must be XxxGatewayFilterFactory, config writes Xxx; other names cause "Unable to find GatewayFilterFactory".

Body read once then gone. Reading request body in filter causes downstream to receive nothing. Correct approach relies on AdaptCachedBodyGlobalFilter caching, or wrap as re-readable DataBuffer.

CORS configured twice. Gateway has CorsWebFilter, downstream also @CrossOrigin, response shows two Access-Control-Allow-Origin, browser errors. CORS only at gateway.

WebSocket route missing ws:// prefix. Using lb:// for WebSocket fails at upgrade handshake; must write lb:ws://service-id.

Ignoring transfer-encoding. Downstream returns chunked but gateway aggregates, large responses fully buffered in memory. File download interfaces should configure streaming pass-through.

Troubleshooting Three Axes: ① Enable logging.level.org.springframework.cloud.gateway=TRACE to see each route's match result and actual forward URL; ② Use /actuator/gateway/routes endpoint to verify runtime route table (reflects final in-memory state, not YAML original); ③ Monitor reactor.netty connection metrics, watch active connections and pending acquire for saturation.

9. Learning Path & Interview Questions

9.1 Four-Stage Roadmap

Beginner (1 week): Run one lb:// route, master Path/Method/Header predicates and StripPrefix/AddRequestHeader filters.

Advanced (2 weeks): Write GlobalFilter for auth/logging, understand order and onion model; integrate RequestRateLimiter for per-user rate limiting; attach Sentinel or Resilience4j to routes for circuit breaking.

Deep (2 weeks): Read four source sections—lookupRoute, FilteringWebHandler, DefaultGatewayFilterChain, NettyRoutingFilter; clarify lb:// URL rebuild timing.

Practice (ongoing): Conduct gateway stress test (wrk or JMeter), observe EventLoop blocking, connection pool queueing, off-heap memory curves; implement timeout pyramid in config and validate.

9.2 Twelve Interview Questions (with Answers)

Q1: Gateway's underlying principle? Based on WebFlux + Reactor Netty async non-blocking gateway. Request enters, RoutePredicateHandlerMapping traverses routes for predicate match, first match wins; merges route-level and global filters, sorts, hands to FilteringWebHandler to form chain; chain tail NettyRoutingFilter uses HttpClient to forward, NettyWriteResponseFilter writes response back. Entire process non-blocking, driven by Reactor pub/sub.

Q2: Why WebFlux not Servlet? Connection-thread relationship decides. Servlet one-request-one-thread, 200 threads support 200 concurrent; gateway as full-site entry has tens of thousands connections, thread model collapses. WebFlux uses few EventLoop threads for massive connections, thread immediately serves other requests while waiting downstream. Cost: zero blocking allowed on chain, ThreadLocal and MDC fail.

Q3: Route, Predicate, Filter relationship? Route is minimal forwarding unit containing id, target URI, predicate set, filter set; Predicate decides if request hits this route, multiple predicates AND; Filter processes request/response after match. One sentence: Predicate decides where, Filter decides how.

Q4: How are filter chain pre/post implemented? Via recursion not loop. DefaultGatewayFilterChain creates new index+1 chain object each call, passes "remaining chain" to current filter. Code before chain.filter() is pre, after is post. Recursion goes to tail then returns layer by layer, return path naturally reverse—onion model.

Q5: Global vs route-level filter who runs first? Not necessarily. Both merged into same list in FilteringWebHandler, sorted by AnnotationAwareOrderComparator, smaller order first. So "global first" is illusion; auth global filter must explicitly use negative order to guarantee earliest.

Q6: Where is lb:// replaced with real IP? In filter chain at ReactiveLoadBalancerClientFilter (order ~10150). It extracts serviceId, asks LoadBalancer for instance, rewrites URL to http://ip:port, writes to GATEWAY_REQUEST_URL_ATTR, then NettyRoutingFilter gets usable address. Shows load balancer filter not only picks machine but also rebuilds URL.

Q7: How to configure gateway timeout reasonably? Layered decreasing: gateway response-timeout < caller (frontend/Nginx) timeout, > downstream Feign timeout. Typical: Nginx 10s > Gateway 3s > Feign 2s > Business 1.5s > DB 800ms. Also connect-timeout separate ~1s, pool acquire-timeout must compress from default 45s to seconds, else downstream slow causes queue buildup in pool.

Q8: Can gateway have blocking code? What happens? No. EventLoop threads only ~2×CPU cores; one blocked, thousands of connections on it queue, symptom: CPU not high but all requests hang. Must call blocking interface, use Mono.fromCallable + subscribeOn(Schedulers.boundedElastic()) to switch to elastic pool, but cannot abuse or gateway degrades back to blocking model.

Q9: RequestRateLimiter principle? Parameter meaning? Redis-based token bucket, Lua for atomicity. replenishRate = tokens per second, burstCapacity = bucket capacity (instant burst allowance); equal = steady, burst larger enables burst. Granularity via KeyResolver: IP misinjures NAT users, constant = global limit, production uses user ID preferred, fallback IP composite key.

Q10: Custom filter considerations? Three things: Class name must be XxxGatewayFilterFactory, config writes Xxx only, suffix auto; order explicit—auth negative, metrics 0, response transform LOWEST_PRECEDENCE-1; don't read request body at this stage, reading loses it for downstream, need AdaptCachedBodyGlobalFilter cache mechanism.

Q11: How does gateway pass user context downstream? In GlobalFilter parse token for user info, use request.mutate().header() to write user ID into header for downstream, downstream reads via interceptor. Don't use ThreadLocal or MDC—reactive model may run on different thread, context lost. For cross-filter sharing use exchange attributes or Reactor Context.

Q12: Gateway vs Nginx relationship? Division? Not replacement. Nginx/APISIX excel at Layer 4 proxy, static resources, WAF, ultra-high concurrency, suitable as outermost entry; Gateway strength is seamless Spring ecosystem, reuses service discovery, Security, Micrometer, suitable as unified business microservice entry for routing/auth. Common production combo: outer APISIX shields traffic, inner Gateway manages business.

9.3 One-Page Cheat Sheet

Three abstractions: Route where / Predicate match
    / Filter transform
Match rule: All predicates true, first wins, order=priority
Chain model: Merge two levels -> order sort -> recursion
    pre forward, post reverse (onion)
Key orders: LB 10150 / Forward MAX
    / Write response -1 / Metrics 0
Blocking forbidden: EventLoop no block
    Must block -> boundedElastic
Timeout decreasing: Gateway 3s > Feign 2s > DB 800ms
Pool backpressure seconds: acquire 45s too dangerous
Memory two pits: Off-heap buffer / Request body cache
Rate limit granularity: User first, fallback IP
Auth three actions: Verify / Short-circuit setComplete
    / Header propagate

Summary: Gateway complexity lies not in configuration but in model. Once you accept "entire chain cannot block" as premise, filter order, connection pool, timeout layers, off-heap memory—these seemingly scattered tuning points chain into a single causal link: because non-blocking, threads few, so cannot block; because connections shared, queue timeout must be short; because memory scales with concurrency, body cannot be cached arbitrarily. Next article enters traffic governance core: "Sentinel Flow Control & Circuit Breaking—Sliding Window, Token Bucket & Circuit Breaker State Machine".