Spring Cloud Gateway Auth: OAuth2, JWT, and the Layer You Can't Bypass

Introduction

The most overrated statement in microservices is "the gateway handles authentication." It often only protects the front door: internal direct connections bypass the gateway, downstream services trust a forgeable header, revoked tokens remain valid for hours, and signature verification cannot prevent horizontal privilege escalation. This article covers three core topics: the boundary between authentication and authorization, reconciling stateless tokens with immediate revocation, and why downstream services should trust the gateway.

Three Real Incidents

Incident 1: Gateway Validates JWT, Yet Security Scan Reports Privilege Escalation

The gateway's SecurityWebFilterChain was meticulously implemented, but service-to-service Feign calls connected directly to instance IPs, bypassing the gateway entirely. Downstream service ports were accessible within the internal network — anyone with internal network access could call interfaces directly without a token. Gateway authentication protects only "traffic coming from outside," not "traffic already inside."

Incident 2: Banned Account Continues Placing Orders for Two Hours

Operations banned a fraudulent account, but the token's exp claim still had 110 minutes remaining. JWT is self-contained — issued takes effect immediately, expires only at expiry, and the server has no "revoke" action. This is the cost of statelessness, not a bug. A temporary Redis blacklist was added but lacked TTL configuration, resulting in four million permanent keys after a week.

Incident 3: Gateway Signature Verification Passes, Downstream Accepts Fake Identity

The gateway placed userId into X-User-Id header and forwarded it; downstream used getHeader directly as identity. A tester used Postman to call downstream directly with X-User-Id: 1 and retrieved an admin's order list. Any header the client can write can never serve as identity — unless you can prove it hasn't been tampered with.

Technical Analysis: Three Terms, Two Boundaries

2.1 Authentication, Authorization, and "Jianquan"

Authentication (AuthN): Who are you? — Signature verification / database lookup / password comparison.

Authorization (AuthZ): What can you do? — Scope / role / resource ownership.

Jianquan (鉴权): Colloquial Chinese term mixing both.

Most "privilege escalation vulnerabilities" are actually missing authorization , not authentication failures. Authentication only guarantees "this request indeed comes from user 1001"; it cannot guarantee "1001 cannot view 2002's order." The latter requires business-layer resource ownership checks — the gateway cannot intervene because it doesn't know who owns order 8888.

2.2 Four Session Forms, Only Trade-offs

Session: Server-side storage, immediate revocation possible. Multi-instance requires sharing, scaling has cost.

JWT: Self-contained, verify signature, stateless. Cannot revoke, large payload, claims become stale.

Opaque token: Random string, introspection per request. Immediate revocation possible, adds one RPC.

Gateway token exchange: External token → internal credential. Clear boundary, extra maintenance layer.

These four sit on the same axis: immediate revocation requires state; statelessness requires accepting revocation latency. Architecture selection answers: "How long a revocation window can you tolerate?" Finance may require seconds; content communities can accept two hours.

2.3 Critical Misconception: JWT Is Not Encryption

Three Base64URL segments; the middle segment is readable on any decoder site. Do not put phone numbers, ID cards, balances, internal IPs into JWT. Seen cases where entire permission lists were stuffed into payload, making tokens 8KB and hitting header size limits — neither secure nor efficient. JWT guarantees "not tampered with" (signature), not "invisible to others" (encryption).

2.4 Core Contradiction: Statelessness vs Immediate Revocation

Three reconciliation methods (detailed in 6.1) — short token + refresh, blacklist, version number — share the same idea: downgrade "query on every request" to "query only when necessary." Understanding this stops the debate "can JWT be revoked?" — yes, but you accept it's no longer purely stateless.

2.5 A Model: Four Questions Every Credential System Must Answer

Who signed it? iss + signature — prevents forgery.

For whom? aud — prevents cross-system replay.

What can they do? scope/role — authorization scope.

How long valid? exp + revocation mechanism — timeliness.

Use these four questions to audit any solution: only verifying signature but not aud allows System C to use System A's token against System B; having exp but no revocation means banning requires waiting for expiry; having scope but no resource ownership equals horizontal privilege escalation. Missing one question = a known attack surface.

2.6 What OAuth2 Is and Isn't

OAuth2 is an authorization framework , not an authentication protocol. It solves "how a third-party app gets limited access without user password." Using OAuth2 for login (treating access token as identity credential) is common practice but know the cost: access token semantics are "authorized to access a resource," not "proves who you are," which is why OIDC added id_token. Engineering rule: If you only need "login," use OIDC; if you need "let third party access user's resources," use OAuth2. They often coexist but must not be conceptually confused.

Core Flow: How a Request Passes the Gate

3.1 Seven Steps

Client sends Authorization: Bearer xxx

Gateway filter extracts header

Verify signature (local public key / JWKS)

Validate claims: exp, nbf, iss, aud

Build security context

First delete client-supplied same-name headers

Inject internal identity headers, forward

Step 6 is the security key and most often missed. Order must be delete-then-write ; otherwise client-supplied X-User-Id impersonates directly. Safer approach: globally configure RemoveRequestHeader in gateway default-filters to clear all identity headers first, then have filter inject — far more reliable than per-route configuration.

3.2 Authorization Code + PKCE: Five Actions

Client generates code_verifier (random), challenge = BASE64URL(SHA256(v))

Redirect to /oauth2/authorize?code_challenge=...&method=S256

User logs in and authorizes, redirect returns with code

Exchange code + verifier for token

Use access_token to call gateway

PKCE solves "public clients cannot securely store secret." SPAs and mobile apps are public clients; embedding secret in package equals public exposure — reverse engineering reveals it. PKCE replaces secret with a one-time verifier: even if code is intercepted, without verifier no token can be obtained. This is now standard for public clients, not optional.

3.3 Service-to-Service Calls: Pass-through vs Token Exchange

Pass-through: Service A calls B with user token. Simple, complete permission chain, but token permissions are excessive.

Exchange: Use client_credentials to exchange. Least privilege, auditable, but loses "on behalf of which user."

Recommended: Dual identity — user identity passed through + service identity separately signed.

Most robust is dual identity : one header carries user (gateway-signed internal credential), another carries service (mTLS or independently issued). Downstream decides per interface nature — config queries trust only service identity; user data queries require both. Pass-through alone gives B a credential "can act as user for anything"; if B is compromised, attack surface equals user's full permissions.

3.4 Three-Layer Authorization, Each Handles a Segment

Gateway layer (coarse): Token present? Scope sufficient? Blocks 90% invalid traffic.

Service layer (medium): Can this role call this interface? @PreAuthorize method-level.

Data layer (fine): Does this data belong to him? WHERE clause forces user_id.

Clarifying responsibilities solves the common debate "where should this check live?": Decidable from token fields → gateway; requires DB lookup → service; requires data ownership → data layer. Anti-pattern: forcing fine-grained checks into gateway (e.g., gateway queries DB "does this user have permission for this order?") — gateway becomes coupling point for all business, permission logic changes require gateway redeploy, and each request adds a DB call, turning the lightest component into the heaviest.

3.5 Refresh and Concurrent Refresh

401 → use refresh_token to obtain new. Concurrent 10 requests all 401 → 10 refresh attempts, last 9 fail because refresh token rotated. Solution: single-flight — in-process lock, refresh once, other requests wait for result. This is a client-side must-have: with rotation enabled, old refresh token invalidates immediately after use; concurrent refresh causes "refresh succeeded but kicked self offline" — user experiences mysterious logout, only under weak network or high first-screen concurrency. Making refresh single-flight is a required SDK capability, not optional optimization.

Core Source Code: Verification, Mapping, Forwarding

4.1 Five Steps of Verification, Third Is Critical

// Simplified from NimbusReactiveJwtDecoder
// 1 Parse header, get alg / kid
// 2 Find public key by kid (cache JWKS)
// 3 Verify signature with alg
// Algorithm MUST be server-side whitelist
// 4 Validate exp / nbf / iss / aud
// 5 Convert to Jwt object

Step 3 is the security watershed: Algorithm must be fixed by server, never trust header. Two most famous JWT vulnerabilities originated here — "alg: none" (removes signature, server follows header and passes), "RS256 downgraded to HS256" (uses public key as HMAC secret, attacker with public key can sign). Nimbus rejects none by default, but custom parsers often fall for it.

4.2 Claim-to-Authority Mapping

// JwtGrantedAuthoritiesConverter
// Default reads scope / scp claims
// Each prefixed with SCOPE_
// scope: order.read → SCOPE_order.read
// Custom: read roles → ROLE_ prefix

Default behavior often mismatches expectations: scope contains order.read, but authority becomes SCOPE_order.read , so hasRole("ADMIN") always false while token clearly has role. Either customize converter to map roles to ROLE_ prefix, or standardize project-wide on hasAuthority("SCOPE_xxx") — mixing both styles is the most error-prone approach.

4.3 Where Authentication Filter Should Sit

RemoveCachedBodyFilter MIN
AuthGlobalFilter -100
RouteToRequestUrlFilter 10000
ReactiveLoadBalancerClient 10150
NettyRoutingFilter MAX

Auth filter must precede RouteToRequestUrlFilter — it modifies headers, then target URL is built, then instance selected. Placed after, URL is already fixed, header changes useless. Also must not precede RemoveCachedBodyFilter; though auth usually reads only headers, if body reading for signature verification is needed, body would be unavailable. Use negative order with explicit gaps — more upgrade-resistant than pinning exact numbers.

4.4 Token Issuance: Only Correct Place for Custom Claims

@Bean
OAuth2TokenCustomizer<JwtEncodingContext> customizer() {
    return ctx -> {
        if (OAuth2TokenType.ACCESS_TOKEN.equals(ctx.getTokenType())) {
            ctx.getClaims().claim("tenant",
                ctx.getPrincipal().getAttribute("tenantId"));
        }
    };
}

Custom claims belong here; never manually concatenate JWT strings in filters. Also realize: once issued, claims are fixed — user changes tenant or gains role, old token retains old values until expiry. This is another manifestation of the 2.4 contradiction, and the real reason "permission changes take time to take effect."

4.5 Introspection: Only in Two Scenarios

Introspection (RFC 7662) calls authorization server per request. Pros: immediate revocation. Cons: extra RPC per request, auth server shifts from "used at issuance" to "depended on per request," becoming hard single point. Applicable only when: token requires second-level revocation (finance, risk control, classified) or token is opaque (third party gives only random string). Otherwise local verification is faster, more stable, cheaper — 0.1ms vs 2ms, 20x difference.

4.6 Ready-to-Use Authentication Filter

public Mono<Void> filter(
    ServerWebExchange ex,
    GatewayFilterChain chain) {
    String h = ex.getRequest().getHeaders()
        .getFirst(HttpHeaders.AUTHORIZATION);
    if (h == null || !h.startsWith("Bearer ")) {
        return reject(ex, 401);
    }
    Jwt jwt;
    try {
        jwt = decoder.decode(h.substring(7));
    } catch (JwtException e) {
        return reject(ex, 401);
    }
    String uid = jwt.getSubject();
    String sig = signer.sign(uid, now, nonce);
    ServerHttpRequest req = ex.getRequest()
        .mutate()
        .headers(hh -> {
            hh.remove("X-User-Id");
            hh.remove("X-User-Sig");
            hh.set("X-User-Id", uid);
            hh.set("X-User-Sig", sig);
        })
        .build();
    return chain.filter(ex.mutate().request(req).build());
}

Four header operations are the core: remove then set. Without remove, client-supplied values persist — header is collection semantics; set may overwrite all same-name values or append one, implementation-dependent, cannot gamble. Explicit delete is the only reliable way. Two more details: verification failure uniformly returns 401, not distinguishing "expired" from "signature error" — distinguishing tells attacker "your token structure is correct, just expired," narrowing search space; internal signature includes timestamp and nonce so downstream can replay-protect, otherwise captured packet replayable forever.

Production Configuration: A Deployable Security Chain

5.1 SecurityWebFilterChain

@Bean
SecurityWebFilterChain chain(ServerHttpSecurity http) {
    return http
        .csrf(ServerHttpSecurity.CsrfSpec::disable)
        .httpBasic(ServerHttpSecurity.HttpBasicSpec::disable)
        .formLogin(ServerHttpSecurity.FormLoginSpec::disable)
        .authorizeExchange(ex -> ex
            .pathMatchers("/actuator/health").permitAll()
            .pathMatchers("/api/public/**").permitAll()
            .anyExchange().authenticated())
        .oauth2ResourceServer(o -> o.jwt(Customizer.withDefaults()))
        .build();
}

Three disables are mandatory: gateway is pure resource server, no form login, no Basic, no CSRF. But CSRF disable is conditional: Bearer token in header is naturally CSRF-immune (browser won't auto-attach custom cross-origin header), so can disable; once client puts token in cookie, CSRF must be enabled — browser auto-sends it, CSRF becomes real. This judgment cannot be blanket "we're frontend-backend separated."

5.2 Claim Design

sub: User ID, unique, immutable

iss: Issuer, must validate — prevents cross-system replay

aud: Audience, critical when multiple systems share key — only isolation

exp: Expiry, 15-30 minutes

nbf: Not-before, validate if present

jti: Unique ID, blacklist depends on it

scope: Permissions, coarse-grained

tenant: Tenant ID, mandatory for multi-tenancy

Two details deserve emphasis. aud: Without validation, System A's token works on System B, both verify signature successfully — most covert privilege escalation. jti: Only meaningful for blacklist; otherwise wastes token bytes, every request transmits extra dozens of bytes.

5.3 Keys: Why Production Almost Always Uses RS256

HS256: Symmetric, one key. Fast, but all verifiers hold signing capability. Leak = total collapse.

RS256: Asymmetric, private key signs only. Verifiers get public key. Public key is public anyway, leak irrelevant.

JWKS: Public key set with kid. Old and new keys coexist, supports smooth rotation.

HS256's issue isn't algorithm weakness but key distribution model : every verifying service needs the key, and possessing key equals signing capability. One leak among ten services collapses entire system. RS256 separates "sign" and "verify"; verifiers only hold public key, leak harmless. Rotation hard rule: old and new public keys must coexist in JWKS for at least one token lifetime. Keeping only new key instantly fails all still-valid old tokens — entire site users simultaneously kicked offline, hard to trace to key rotation ten minutes prior.

5.4 Clock Skew: Never Set to Zero

JWT validation defaults to 60s clock skew. Multi-datacenter deployment setting this to 0 is a classic error — NTP jitter, container migration, VM suspend/resume cause millisecond exp misjudgment, resulting in intermittent, random, hard-to-reproduce verification failures — the hardest fault class to debug. If tightening, order matters: first ensure cluster-wide NTP sync with millisecond skew, then lower to 5-10s. Reverse order = using production as clock laboratory.

5.5 Three Authorization Server Defaults, All Traps

accessTokenTimeToLive 5m

refreshTokenTimeToLive 60m

reuseRefreshTokens true

authorizationCodeTTL 5m

All three are counter-intuitive. reuseRefreshTokens defaults to true — means no rotation; refresh returns same refresh token. Contradicts 3.5 rotation: you think rotation enabled, but it's been reusing. To rotate must explicitly set false, and client must implement single-flight refresh, else concurrent refresh kicks each other. Other two: access token 5 minutes too short for internal systems, high-frequency calls keep client refreshing, amplifying auth server load; refresh token 60 minutes too short for B2C, users login multiple times daily. Both must be tuned to business rhythm, not defaults.

Five Advanced Topics

6.1 Three Ways to Make Token Immediately Invalid

Blacklist: Redis stores jti, TTL = remaining validity. Query per request (local cache 5s).

Short token + refresh: Access 5-15 min. Revoke = stop issuing refresh.

Version number: User table stores tokenVersion. Password change increments +1, mismatch rejects.

Difference is granularity: blacklist revokes single token (single device logout), version number revokes whole user (password change, ban, force logout all devices), short token most elegant but revocation has latency — max one access token lifetime. Production typically combines: access 15 min + refresh 30 days, ban uses version number, single-device logout uses blacklist. Don't expect one mechanism to solve all scenarios.

6.2 Why Downstream Should Trust Gateway

Network isolation: Service ports open only to gateway. Strongest, requires proper network planning.

Internal signature header: Gateway signs with internal key {uid, ts, nonce}. Downstream verifies + replay protection.

mTLS: Mutual certs, service identity provable. Most thorough, highest ops cost.

Most common reality: "network isolation + plaintext header" relying on "outsiders can't reach." Works in well-planned single cluster but assumption fragile: one exposed debug port, one misconfigured NodePort, one temporary jump host collapses it. For internet-facing services, this equals zero protection. Judgment is simple: if this header can be forged, it's not identity, just a hint. Either sign to make it unforgeable, or network-isolate to make it unreachable — no third option.

6.3 Horizontal Privilege Escalation: The Half Gateway Can't Reach

Gateway confirms sub=1001, but doesn't know if order 8888 belongs to 1001 — such decisions belong to business layer. And must be done at data access layer , not interface layer. where user_id = ? is far more reliable than if (order.userId == current) because the latter gets bypassed by "adding a new query interface": developer forgets the if, vulnerability appears; SQL where clause auto-applies to every query. Put permissions in query conditions, not if checks — most effective engineering habit against horizontal escalation.

6.4 Attack Checklist: Five Must-Know

alg=none: Removes signature; if server follows header, passes.

Algorithm confusion: RS256 public key used as HS256 secret; attacker signs themselves.

kid injection: kid concatenated into file path; arbitrary file read.

Replay: Capture and resend; mitigate with jti/nonce or short exp.

Token leakage: Logs print full token; Referer leaks it.

Fifth is most prevalent and unfair: log.info("token: {}", token) — one line writes credentials into log system; log access usually looser than DB, one ELK account searches all site tokens. Rule: token only allowed in Authorization header. Not in logs, not in URL params, not in cookies (unless CSRF enabled). URL especially dangerous — enters Nginx access log, browser history, Referer.

6.5 Performance Ledger

Local RS256 verification ~0.1ms

Local HS256 verification ~0.02ms

Introspection RPC 0.5-3ms

JWKS fetch: once at startup, then cached; refresh only on kid miss

Conclusion clear: local verification overhead negligible; introspection is real cost — at 5000 QPS, introspection means 5000 RPC/sec to auth server, which must handle that load. "Switch to introspection for security" needs explicit reason (second-level revocation), not "feels safer."

6.6 SSO and Multi-Device Logout

SSO essence: "centralized authentication, distributed sessions": auth server holds single login state, each app holds own token. Inevitable result: logout is either single-device or incomplete.

Single-device logout: Clear local token + blacklist this jti. Other devices unaffected.

All-device logout: Version number +1 (simplest) or enumerate all user jti and blacklist.

SSO global logout: Auth server clears session; apps rely on token expiry. Window up to one full lifetime.

Last line misunderstood: click "logout all devices" in SSO, some app still works minutes — not bug, already-issued tokens cannot be recalled. Shorten window only two ways: reduce access token lifetime, or introduce blacklist/version verification. No third way.

6.7 Multi-Environment Isolation: Test Tokens Must Not Hit Production

Real high-frequency scenario: developer gets token in test, local debug accidentally points to production gateway — works because both use same key, same iss, even same aud. Isolation requires three things, none optional: keys isolated per environment (test private key must never sign token production accepts), iss or aud carries environment tag (production only accepts iss=prod), JWKS URL not shared across environments (most overlooked, config often uses single global constant).

Eight High-Frequency Pitfalls

① Only protecting gateway entry — internal direct connect bypasses, equals no auth.

② Downstream trusts client header — X-User-Id forgeable.

③ Forgot delete-then-write — client-supplied same-name header survives.

④ Algorithm follows header — accepts none / allows downgrade.

⑤ Token logged — most common credential leak path.

⑥ Role change not effective — claim fixed at issuance.

⑦ JWKS rotation without overlap — entire site instantly kicked offline.

⑧ CSRF disabled blindly — but token placed in cookie.

Among these, ①②③ are "wrong architectural assumptions" — fixing requires network and protocol changes; ④⑤ are "code details" — quick fix but high impact; ⑥⑦⑧ are "ops pitfalls" — explode only at specific moments, invisible normally. Self-audit by this classification more effective than checklist.

Troubleshooting Three Axes

Confirm request passed gateway: Downstream logs remote addr, check for internal identity header.

Decode token inspect claims: No verification, just decode. exp / aud / scope — are they really what you think?

Distinguish 401 vs 403: 401 = didn't enter door — check gateway verification. 403 = entered but no permission — check converter mapping and business authorization.

Third is most useful clue. 401 means auth failed — token missing, expired, verification failed, iss/aud mismatch; 403 means auth passed but insufficient permission — scope mapping wrong, role prefix mismatch, resource ownership check failed. Two map to completely different code locations; not distinguishing first makes code reading direction probably wrong.

Learning Path & Interview Questions

9.1 Four Stages

Beginner (1 day): Spin up resource server, pass handwritten JWT through "verify + permit", understand what three disables in SecurityWebFilterChain turn off.

Intermediate (1 week): Integrate real auth server (Spring Authorization Server or Keycloak), complete auth code + PKCE, wire custom claims, role mapping, refresh flow.

Deep (2 weeks): Read four source sections — Nimbus verification & algorithm whitelist, GrantedAuthorities mapping, JWKS cache & kid refresh, JwtGenerator issuance; manually reproduce alg=none and algorithm confusion attacks.

Practice (ongoing): Design complete solution: token lifetime, revocation mechanism, service-to-service identity, downstream trust model; run penetration test specifically trying "direct downstream call" and "forge header."

9.2 Twelve Interview Questions (with Summary Answers)

Key Q&A pairs covered: authentication vs authorization distinction; JWT three-part structure and non-encrypted payload; HS256 vs RS256 selection; JWT immediate revocation methods; why downstream needs second defense after gateway; why not replace token with plain X-User-Id; OAuth2 grant types for frontend-backend separation; PKCE problem solved; refresh token rotation causing mysterious logout; introspection vs local verification selection; common JWT attacks; gateway auth performance estimation.

9.3 One-Page Cheat Sheet

Three terms
  AuthN = who you are, AuthZ = what you can do
  90% privilege escalation is missing AuthZ, not AuthN failure
JWT
  Three Base64 parts, signature not encryption
  No sensitive info, no giant permission lists
  Claim fixed at issuance, permission change waits for expiry
Keys
  Multi-service always RS256 + JWKS
  Rotation: old & new public keys coexist one lifetime
  clockSkew never set to 0
Revocation
  Blacklist (precise) / Short token (slow) / Version (coarse)
  Combine all three, don't expect one to solve all
Boundaries
  Gateway only guards front door, internal direct connect needs separate defense
  Identity header: delete then write, best with internal signature
  Horizontal escalation: WHERE clause, not if check
Troubleshooting
  401 = door not entered, check gateway
  403 = entered no permission, check mapping
  Decode claims first, then suspect code

Summary: Gateway authentication difficulty isn't configuration syntax, but clarifying three boundaries — authentication vs authorization boundary (gateway only does former), stateless vs revocation boundary (immediate revocation requires state), gateway vs internal network boundary (front door isn't the only door). Clear these three, rest is config details; unclear, config however perfect only guards the front door.