OAuth 2.0 at Billion-Scale: Unified Auth Center Architecture & Pitfall Guide

1. Why many OAuth 2.0 projects break under load

Teams often treat OAuth 2.0 as a simple login component instead of a high‑concurrency identity infrastructure. While the basic capabilities—login page, token validation at the gateway, user‑ID propagation, and refresh token usage—appear to work, traffic spikes such as flash sales, seckill events, live‑stream spikes, and platform callback floods expose a series of bottlenecks.

All authentication requests are routed back to the auth center, creating a single point of contention.

JWTs cannot be revoked instantly; a user who logs out may still have a valid token.

Concurrent refreshes cause multiple devices to kick each other offline.

Clock drift between gateway, service nodes, and auth center leads to tokens becoming invalid immediately after issuance.

Key rotation without a transition period breaks existing tokens.

Uncontrolled token propagation between micro‑services results in over‑privilege and missing audit trails.

Multi‑region deployments suffer from blacklist and session‑state propagation delays.

These symptoms show that OAuth 2.0 is only the protocol entry point; a production‑grade unified authentication center must solve identity issuance, propagation, verification, revocation, audit, and governance across a distributed system.

2. Production‑grade OAuth 2.0 design

2.1 Defining the protocol boundary

As of 2026‑06‑26 OAuth 2.1 is still an IETF draft, so production continues to rely on OAuth 2.0 plus best‑practice extensions. The recommended grant types are:

Authorization Code + PKCE for user‑facing login.

Client Credentials for service‑to‑service calls.

Deprecate Password Grant and Implicit Grant for new systems.

Enforce additional JWT security checks rather than “signature‑only” validation.

2.2 Why Authorization Code + PKCE is the default answer

PKCE adds a code_verifier that proves the client’s identity during the code exchange, preventing intercepted codes from being reused. It also opens the door for inserting device fingerprints, CAPTCHAs, risk‑based rules, MFA, enterprise SSO, and fine‑grained consent auditing.

2.3 Separating Access Token and Refresh Token responsibilities

Access Token – JWT, high‑frequency verification, stateless, low latency, local signature verification.

Refresh Token – opaque random string or controlled JWT, stateful, revocable, rotatable, stored in a persistent session store.

The most common production combo is:

Access Token as JWT.

Refresh Token as a random string persisted in Redis (or similar).

Reason: high‑frequency checks stay decentralized, while low‑frequency renewal remains fully governed.

2.4 JWT’s core value is moving verification to the edge

JWT is not just “no DB lookup”. Its real engineering value is to shift the heavy verification load from the auth center to gateways and resource services, allowing the auth center to focus on issuance, key rotation, and governance.

The auth center no longer participates in every API request.

Resource paths no longer depend on the auth center’s real‑time availability.

The system evolves from “synchronous coupling” to “central issuance, distributed verification”.

Trade‑off: immediate revocation is impossible; a short‑lived JWT must be complemented by a blacklist with an acceptable consistency window.

2.5 Token lifetime recommendations

Access Token: 10–20 minutes.

Refresh Token: 7–30 days.

Gateway pre‑renewal threshold: 2–5 minutes before expiry.

Refresh Token must be rotated on each use.

Benefits: short‑lived access tokens reduce leak windows; long‑lived refresh tokens preserve user experience; rotation enables replay detection and session governance.

2.6 Blacklist design – near‑real‑time consistency

Because JWTs cannot be invalidated instantly, a hybrid approach is used:

Store revoked jti in Redis.

Cache recent revocations locally in each gateway (Caffeine or similar).

Broadcast revocation events via Kafka to shrink the propagation delay.

High‑risk endpoints may force a Redis lookup instead of relying solely on the local cache.

This balances latency, consistency, and load on the central store.

2.7 Service‑to‑service call models

Model 1 – User context propagation : forward the original user Access Token (or a downstream token derived from it) when a user‑initiated request traverses services (e.g., order → coupon). This preserves the user principal for fine‑grained audit and authorization.

Model 2 – Machine identity : use a dedicated client‑credential token for background jobs, batch tasks, or internal platform calls. This makes the caller’s identity explicit and isolates it from user sessions.

Anti‑patterns to avoid:

Using a single “super‑admin” client for all internal calls.

Blindly forwarding user tokens for every internal request.

Relying on the gateway alone for all authorization decisions.

Correct practice: gateway performs coarse identity checks, each service performs its own resource‑level authorization, and user‑context and machine‑identity flows are kept separate.

3. Core implementation – Spring Authorization Server

3.1 Maven dependencies

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-security</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-oauth2-authorization-server</artifactId>
    </dependency>
    <!-- Redis, Kafka, Jackson, etc. -->
</dependencies>

3.2 Token and security properties

@ConfigurationProperties(prefix = "security.oauth2")
public class OAuthSecurityProperties {
    private String issuer = "https://auth.example.com";
    private Duration accessTokenTtl = Duration.ofMinutes(15);
    private Duration refreshTokenTtl = Duration.ofDays(15);
    private String audience = "gateway";
    private String keyId = "auth-key-2026-06";
    // getters & setters
}

3.3 Registering clients with production‑grade settings

@Bean
public RegisteredClientRepository registeredClientRepository(PasswordEncoder encoder, OAuthSecurityProperties props) {
    RegisteredClient webClient = RegisteredClient.withId(UUID.randomUUID().toString())
        .clientId("mall-web")
        .clientSecret(encoder.encode("replace-with-secret"))
        .clientAuthenticationMethod(ClientAuthenticationMethod.CLIENT_SECRET_BASIC)
        .authorizationGrantType(AuthorizationGrantType.AUTHORIZATION_CODE)
        .authorizationGrantType(AuthorizationGrantType.REFRESH_TOKEN)
        .redirectUri("https://www.example.com/login/oauth2/code/mall-web")
        .postLogoutRedirectUri("https://www.example.com/logout/success")
        .scope("openid")
        .scope("profile")
        .scope("order.read")
        .scope("coupon.write")
        .clientSettings(ClientSettings.builder().requireAuthorizationConsent(false).requireProofKey(true).build())
        .tokenSettings(TokenSettings.builder()
            .accessTokenTimeToLive(props.getAccessTokenTtl())
            .refreshTokenTimeToLive(props.getRefreshTokenTtl())
            .reuseRefreshTokens(false)
            .build())
        .build();

    RegisteredClient jobClient = RegisteredClient.withId(UUID.randomUUID().toString())
        .clientId("inventory-job")
        .clientSecret(encoder.encode("replace-with-job-secret"))
        .clientAuthenticationMethod(ClientAuthenticationMethod.CLIENT_SECRET_BASIC)
        .authorizationGrantType(AuthorizationGrantType.CLIENT_CREDENTIALS)
        .scope("inventory.internal")
        .tokenSettings(TokenSettings.builder().accessTokenTimeToLive(Duration.ofMinutes(10)).build())
        .build();

    return new InMemoryRegisteredClientRepository(webClient, jobClient);
}

3.4 JWK management – never generate keys at startup

Production must load private keys from KMS, Vault, or mounted secrets and publish a stable JWK Set with versioned kid. Historical public keys stay in the set until all tokens signed with them expire.

@Configuration
public class JwkConfig {
    @Bean
    public JWKSource<SecurityContext> jwkSource(OAuthSecurityProperties props, KeyPairProvider provider) {
        KeyPair current = provider.current();
        RSAKey currentKey = new RSAKey.Builder((RSAPublicKey) current.getPublic())
            .privateKey((RSAPrivateKey) current.getPrivate())
            .keyUse(KeyUse.SIGNATURE)
            .algorithm(JWSAlgorithm.RS256)
            .keyID(props.getKeyId())
            .build();
        List<JWK> keys = new ArrayList<>();
        keys.add(currentKey);
        provider.previous().ifPresent(prev -> keys.add(new RSAKey.Builder((RSAPublicKey) prev.getPublic())
            .keyUse(KeyUse.SIGNATURE)
            .algorithm(JWSAlgorithm.RS256)
            .keyID(prev.getKid())
            .build()));
        return new ImmutableJWKSet<>(new JWKSet(keys));
    }
}

3.5 JWT claim hygiene

Only minimal claims needed for verification and authorization should be placed in the JWT: sub – user ID sid – session ID (enables session‑level kick‑out) iss, aud, exp, nbf,

typ

roles

– required role set amr – authentication methods used (e.g., pwd, mfa)

Do NOT embed display fields such as nickname, avatar, or large permission trees; those belong to a user‑profile service.

4. Gateway edge authentication

4.1 Responsibilities of the gateway

Validate Bearer token format.

Perform local JWT signature verification.

Check iss, aud, and typ claims.

Consult a local blacklist cache.

Extract minimal identity headers (user‑id, session‑id, tenant‑id, scopes) for downstream services.

Apply stricter policies on high‑risk endpoints.

4.2 Reactive JWT decoder with multi‑claim validation

@Configuration
public class GatewayJwtConfig {
    @Bean
    public ReactiveJwtDecoder reactiveJwtDecoder(SecurityJwtProperties props) {
        NimbusReactiveJwtDecoder decoder = NimbusReactiveJwtDecoder.withJwkSetUri(props.getJwkSetUri()).build();
        OAuth2TokenValidator<Jwt> validator = new DelegatingOAuth2TokenValidator<>(
            JwtValidators.createDefaultWithIssuer(props.getIssuer()),
            new AudienceValidator(props.getAudience()),
            new TokenTypeValidator("access_token")
        );
        decoder.setJwtValidator(validator);
        return decoder;
    }
}

4.3 Global filter that ties verification, blacklist, and header propagation

@Component
public class JwtAuthenticationFilter implements GlobalFilter, Ordered {
    private final ReactiveJwtDecoder jwtDecoder;
    private final TokenBlacklistService tokenBlacklistService;
    // constructor omitted
    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        String auth = exchange.getRequest().getHeaders().getFirst(HttpHeaders.AUTHORIZATION);
        if (auth == null || !auth.startsWith("Bearer ")) {
            return unauthorized(exchange, "missing_bearer_token");
        }
        String token = auth.substring(7);
        return jwtDecoder.decode(token)
            .flatMap(jwt -> tokenBlacklistService.isBlocked(jwt.getId(), jwt.getExpiresAt())
                .flatMap(blocked -> blocked ? unauthorized(exchange, "token_revoked") : chain.filter(mutateRequest(exchange, jwt))))
            .onErrorResume(e -> unauthorized(exchange, "invalid_token"));
    }
    private ServerWebExchange mutateRequest(ServerWebExchange exchange, Jwt jwt) {
        ServerHttpRequest request = exchange.getRequest().mutate()
            .header("X-Auth-UserId", jwt.getSubject())
            .header("X-Auth-SessionId", claim(jwt, "sid"))
            .header("X-Auth-TenantId", claim(jwt, "tenant_id"))
            .header("X-Auth-Scope", String.join(",", jwt.getClaimAsStringList("scope")))
            .build();
        return exchange.mutate().request(request).build();
    }
    private String claim(Jwt jwt, String key) {
        Object v = jwt.getClaims().get(key);
        return v == null ? "" : v.toString();
    }
    private Mono<Void> unauthorized(ServerWebExchange ex, String code) {
        ex.getResponse().setStatusCode(HttpStatus.UNAUTHORIZED);
        ex.getResponse().getHeaders().add(HttpHeaders.WWW_AUTHENTICATE, "Bearer error=\"" + code + "\"");
        return ex.getResponse().setComplete();
    }
    @Override public int getOrder() { return -200; }
}

4.4 Blacklist service – Redis + local cache + Kafka broadcast

@Service
public class TokenBlacklistService {
    private final ReactiveStringRedisTemplate redisTemplate;
    private final Cache<String, Boolean> localCache = Caffeine.newBuilder()
        .maximumSize(500_000)
        .expireAfterWrite(Duration.ofMinutes(3))
        .build();
    // isBlocked, revoke, Kafka listener omitted for brevity
    private String redisKey(String jti) { return "oauth:blacklist:" + jti; }
}

The design trade‑offs are explicit: local cache gives sub‑millisecond checks but introduces a consistency window; Redis provides strong consistency at the cost of network latency; Kafka shortens the propagation delay for revocation events.

5. Refresh‑Token Rotation – the real security watershed

5.1 Why rotation matters

If a refresh token can be reused indefinitely, its theft grants an attacker a long‑lived credential. Rotation guarantees that each refresh creates a brand‑new token and immediately invalidates the previous one. Re‑use is detected as a replay attack and triggers session‑wide revocation.

5.2 Rotation flow (simplified)

1. Client sends current Refresh Token to the auth server.
2. Server verifies token existence, revocation status, and session association.
3. If valid, the old token is marked used, a new Access Token and Refresh Token are issued, and the new Refresh Token state is persisted.
4. If the old token is presented again, the server treats it as replay, revokes the whole session, and optionally blacklists the Access Token.

5.3 Production‑grade rotation service (pseudocode)

@Service
public class RefreshTokenRotationService {
    private final RefreshSessionRepository repo;
    private final TokenRevocationPublisher publisher;
    @Transactional
    public RotationResult rotate(String refreshTokenId, String clientId, String deviceId) {
        RefreshSession session = repo.findByIdForUpdate(refreshTokenId)
            .orElseThrow(() -> new OAuth2AuthenticationException(OAuth2ErrorCodes.INVALID_GRANT));
        if (!session.clientId().equals(clientId) || !session.deviceId().equals(deviceId)) {
            throw new OAuth2AuthenticationException(OAuth2ErrorCodes.INVALID_CLIENT);
        }
        if (!"ACTIVE".equals(session.status())) {
            repo.revokeBySessionId(session.sessionId(), "REPLAY_DETECTED");
            publisher.publishSessionRevoked(session.sessionId());
            throw new OAuth2AuthenticationException(OAuth2ErrorCodes.INVALID_GRANT);
        }
        repo.markUsed(refreshTokenId, Instant.now());
        RefreshSession next = repo.createNext(session);
        repo.save(next);
        return new RotationResult(next);
    }
}

Key points: use row‑level locking or CAS to avoid concurrent refreshes succeeding with the same old token; on replay, revoke the entire session rather than just the single request.

6. Twelve production pitfalls and mitigations

Treating JWT as an expandable user object – leads to oversized headers and stale claims. Mitigation: keep only verification‑essential fields; fetch mutable user data from a separate profile service.

Verifying only the signature – allows tokens issued for other audiences or types to be accepted. Mitigation: always validate iss, aud, exp, nbf, and typ.

No revocation mechanism – logout or password change does not invalidate existing tokens. Mitigation: maintain a jti blacklist in Redis with local cache and Kafka broadcast.

Refresh tokens not rotated – long‑lived tokens become a single point of compromise. Mitigation: rotate on every use and detect replay.

Key rotation without overlap – new private key instantly breaks verification of existing tokens. Mitigation: keep multiple public keys in the JWK Set and use stable kid identifiers.

Auth center in the hot path – introspection per request creates a bottleneck. Mitigation: move to local JWT verification; keep introspection only for special cases.

Blacklist relying solely on Redis – creates a hotspot under load. Mitigation: add a fast local cache and tiered verification for high‑risk APIs.

Clock drift – tokens appear expired or not yet valid. Mitigation: synchronize all machines via NTP and allow a small clockSkew tolerance.

Blurry multi‑device session boundaries – logout on one device kicks out all devices. Mitigation: embed sid, deviceId, and sessionType in the token and manage them in Redis.

All authorization in the gateway – configuration bloat and inability to express fine‑grained domain rules. Mitigation: let the gateway perform coarse identity checks; let each service enforce its own resource‑level policies.

One policy for admin, open platform, and internal systems – mixes risk levels. Mitigation: separate strategies, TTLs, MFA requirements, scopes, and rate limits per client type.

No audit closure – incidents cannot be traced to user, device, or client. Mitigation: log critical fields ( sub, sid, client_id, device_id, tenant_id, trace_id, result, error_code) and ship them to a centralized observability platform.

7. Real‑world case study: handling a 618 flash‑sale peak

7.1 Background

An e‑commerce platform experienced simultaneous spikes: coupon‑grab login bursts, bulk product publishing by merchants, H5 new‑user logins, and risk‑engine account freezes. Key metrics:

Peak API traffic: 1.2 M QPS.

Login issuance peak: 28 k TPS.

Refresh‑token renewal peak: 42 k TPS.

Blacklist events: 300 k per minute.

7.2 Why the initial design failed

All services performed remote token introspection; logout only cleared cookies; refresh tokens were not rotated; the gateway validated only the signature without aud checks. The result was auth‑center CPU saturation, delayed revocation, and cross‑tenant token misuse.

7.3 Refactored solution

Switch Access Tokens to JWT with local verification.

Enforce full Refresh‑Token rotation stored in Redis.

Gateway now validates iss, aud, and typ.

Redis + Kafka implements near‑real‑time blacklist propagation.

High‑risk APIs force Redis lookups.

Separate client strategies for user‑side, internal jobs, and open platform.

Introduce sid and device identifiers for precise session control.

7.4 Outcome

High‑frequency requests no longer hit the auth center.

Auth center becomes an issuance and governance service rather than a per‑request validator.

Blacklist propagation latency reduced to sub‑second levels.

Refresh‑token replay attacks are detected and trigger session‑wide revocation.

Audits can trace every request back to user, device, client, and tenant.

Key insight: the auth center should be the “control plane” while verification lives at the edge.

8. Containerization & Kubernetes deployment

8.1 Private key handling

Never bake PEM files into the image or store private keys in Git. Load them from KMS, Vault, or Kubernetes Secrets at startup, and rotate via a staged rollout.

8.2 Minimal Dockerfile

FROM eclipse-temurin:17-jre
WORKDIR /app
COPY target/auth-server.jar app.jar
ENTRYPOINT ["java","-XX:+UseContainerSupport","-jar","/app/app.jar"]

8.3 Sample Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: auth-server
spec:
  replicas: 3
  selector:
    matchLabels:
      app: auth-server
  template:
    metadata:
      labels:
        app: auth-server
    spec:
      containers:
      - name: auth-server
        image: registry.example.com/auth-server:2026.06.26
        ports:
        - containerPort: 9000
        env:
        - name: OAUTH_ISSUER
          value: https://auth.example.com
        - name: JWT_PRIVATE_KEY_PATH
          value: /etc/keys/current.pem
        volumeMounts:
        - name: jwt-key
          mountPath: /etc/keys
          readOnly: true
        readinessProbe:
          httpGet:
            path: /actuator/health/readiness
            port: 9000
          initialDelaySeconds: 20
          periodSeconds: 5
      volumes:
      - name: jwt-key
        secret:
          secretName: auth-jwt-private-key

8.4 HPA focus

After moving verification to the edge, horizontal pod autoscaling should monitor issuance‑related metrics (login TPS, token‑signing TPS, refresh‑rotation TPS, Redis latency) instead of total API QPS.

9. Observability & audit

9.1 Mandatory log fields

trace_id

user_id

session_id

client_id

grant_type

token_jti

device_id

tenant_id

result

error_code

9.2 Key metrics

Auth center : login success rate, token‑issuance TPS, refresh‑rotation TPS, client‑level failure rate, Redis/Kafka latency, MFA service latency.

Gateway : JWT verification latency, blacklist cache hit ratio, Redis blacklist query latency, 401/403 ratios, client‑level reject rate.

9.3 Alerting examples

Sudden surge of 401 responses for a specific client.

Spike in refresh‑token replay detections.

Rapid increase in blacklist events for a tenant.

Public‑key fetch failures.

Auth‑center token‑issuance success rate drop.

Redis blacklist query P99 latency breach.

10. Pre‑deployment checklist

10.1 Protocol & security

Default to Authorization Code + PKCE.

Disable Password Grant for new services.

Validate iss, aud, and typ on every verification.

Integrate MFA / risk engine for high‑risk operations.

Enable Refresh‑Token Rotation.

10.2 State & consistency

Implement jti blacklist with Redis + local cache + Kafka broadcast.

Design sid session identifiers for precise session control.

Support single‑device, single‑session, and full‑account logout flows.

Define acceptable revocation propagation latency.

10.3 Key management

Private keys sourced from KMS/Vault/Secrets.

Stable kid values and multi‑key JWK Set for graceful rotation.

Key‑rotation rehearsal plan.

10.4 High‑concurrency & stability

Load‑test login, refresh, revocation, and risk‑engine peaks.

Validate gateway JWT verification latency.

Stress Redis blacklist hot‑spots.

Define clear fallback and degradation strategies.

10.5 Audit & compliance

Ensure traceability to client, user, device, and session.

Separate audit streams for admin/back‑office vs. user‑facing actions.

Mask sensitive data in logs.

11. Conclusion

The most common mistake is treating a unified authentication center as merely a login module. In a billion‑scale system it becomes the identity traffic backbone, responsible for who can log in, who can access which resources, which service acts on behalf of which user, when identities become invalid, and how risk is contained.

Four essential questions must be answered:

How to offload high‑frequency verification from the auth center?

How to retain revocation and session governance while using stateless access tokens?

How to keep user, machine, and session boundaries clear?

How to keep the system operable under high load, rotation, failures, gray‑deploys, and disaster recovery?

Answers:

Protocol: Authorization Code + PKCE, short‑lived JWT, rotatable Refresh Token.

Architecture: central issuance, edge verification, stateful governance.

Engineering: blacklist, session indexes, event broadcasting, key rotation, risk‑layered checks, full audit trail.

Evolution: start with gateway‑centric verification, progress to fine‑grained service authorization, then to sidecar/mesh offload, finally to a zero‑trust identity fabric.

When these aspects are fully considered, OAuth 2.0 transforms from a simple login protocol into the foundation of a secure, high‑performance identity infrastructure capable of supporting billion‑level traffic.

References

Spring Authorization Server Reference

IETF OAuth 2.1 Draft

RFC 8725: JSON Web Token Best Current Practices

RFC 7636: Proof Key for Code Exchange by OAuth Public Clients