Why Every Microservice Architecture Needs an API Gateway

Why a Gateway Is Essential

When a monolith is split into dozens of services, each service gets its own port, authentication logic, rate‑limiting switch and log format. A client request for an order may need to call member, order and product services, all directly exposed to the public network, creating a chaotic architecture.

The gateway places a unified entry point in front of all services, handling authentication, routing, rate‑limiting, logging and other cross‑cutting concerns that are unrelated to business logic.

Problems Without a Gateway

1. Repeated Authentication Logic

Each of the 20 services must parse JWT, verify signatures and check a blacklist, resulting in duplicated code that must be updated in every service when the logic changes, creating security gaps.

Sharing a common library inflates every service’s JAR and forces a full rebuild of all dependent services for any change.

2. Scattered Cross‑Cutting Concerns

Logging, rate‑limiting and CORS are implemented differently across teams (JSON vs plain text logs, Guava RateLimiter vs custom sliding window), making global configuration and troubleshooting difficult.

3. Lack of Global Operational View

Features such as gray releases, A/B testing or global request blocking require modifications in every service; a gateway allows a single configuration to take effect globally.

4. Nginx Is Not a Full Replacement

Nginx excels at static routing but cannot perform JWT parsing, circuit breaking or dynamic routing from a service registry. Configuration changes require reloads, which is impractical for frequent updates.

What a Gateway Is

A gateway consists of two capabilities: gateway = routing + filter‑chain Routing: Receives external requests and forwards them to the appropriate microservice based on path, headers or parameters.

Filter Chain: A series of filters that process requests on entry and exit, handling authentication, rate‑limiting, logging, circuit breaking, CORS, etc.

Core Responsibilities

1. Unified Authentication

The gateway parses the JWT token, checks the Redis blacklist, extracts user information and injects it into request headers before forwarding.

// Extract token from request header
String token = request.getHeaders().getFirst("Authorization");
// Check blacklist in Redis
Mono<Boolean> logoutInRedis = reactiveStringRedisTemplate
    .opsForValue()
    .get(LOGOUT_REDIS_KEY_PREFIX + pureToken)
    .map(s -> true)
    .switchIfEmpty(Mono.just(false));
// Parse JWT
Jws<Claims> claimsJws = Jwts.parser()
    .setSigningKey(secretBase64)
    .parseClaimsJws(pureToken);
Claims body = claimsJws.getBody();
String subject = body.getSubject();
String channel = body.get("channel", String.class);

After validation, the gateway removes any external custom headers and injects the verified values:

// Clean possible forged headers
httpHeaders.remove("X-Subject");
httpHeaders.remove("X-Channel");
httpHeaders.remove("X-Primary-Account-Id");
// Inject parsed user info
requestBuilder.header("X-Subject", subject);
requestBuilder.header("X-Channel", channel);
requestBuilder.header("X-Primary-Account-Id", primaryAccountId);

Security note: the gateway must clear external X-Subject (and similar) headers before injecting its own values to prevent header spoofing.

2. Routing and Load Balancing

Clients only need the gateway address; the gateway resolves service instances from a registry (e.g., Nacos/Eureka) and performs load balancing.

spring:
  cloud:
    gateway:
      routes:
        - id: user-service
          uri: lb://user-service   # lb:// uses the service registry for load balancing
          predicates:
            - Path=/api/user-service/**
        - id: order-service
          uri: lb://order-service
          predicates:
            - Path=/api/order-service/**
        - id: pay-service
          uri: lb://pay-service
          predicates:
            - Path=/api/pay-service/**

Backend services can scale up or down without client changes. The gateway also supports API aggregation, sending parallel requests to multiple services and merging the results.

3. Rate Limiting and Circuit Breaking

Rate limiting is implemented per‑API using a Redis + Lua token‑bucket algorithm, allowing fine‑grained QPS control.

-- Get current token count
local last_tokens = tonumber(redis.call("get", tokens_key))
if last_tokens == nil then
  last_tokens = capacity
end
-- Refill tokens based on elapsed time
local delta = math.max(0, now - last_refreshed)
local filled_tokens = math.min(capacity, last_tokens + (delta * rate))
-- Allow if enough tokens
local allowed = filled_tokens >= requested

Using Lua ensures atomic execution in Redis, avoiding race conditions and offering better performance than application‑level distributed locks.

Circuit breaking (e.g., Hystrix) isolates failures per URL, preventing a faulty downstream service from cascading.

// Initialise circuit breaker per URL
Setter setter = Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey(getClass().getSimpleName()))
    .andCommandKey(HystrixCommandKey.Factory.asKey(uriAsCommandKey));
// Publish event when circuit opens
if (command.isCircuitBreakerOpen()) {
  circuitBreakerOpenEventBus.post(command);
}

Sentinel (Spring Cloud Alibaba) provides a visual console for real‑time rule updates without redeployment.

4. Logging and Tracing

The gateway captures request and response logs asynchronously via MQ, avoiding main‑thread blocking.

// Request log → MQ async send
accessLogService.logRequestBody(exchange, bodyStr);
// Response log → MQ async send
accessLogService.logResponseBody(exchange, responseBody);

It also injects a TraceId into response headers, enabling end‑to‑end tracing with Zipkin/Sleuth.

Gateway Landscape Comparison

Nginx – Proven, high‑performance, stable; requires configuration reload and lacks business‑logic handling.

Spring Cloud Gateway – Spring ecosystem friendly, highly extensible; community resources relatively fewer.

Kong – Open‑source API gateway pioneer, large user base; depends on PostgreSQL and performance degrades with >1000 routes.

Apache APISIX – Cloud‑native, excellent performance, multi‑language plugins; documentation still maturing.

Envoy – CNCF graduated project, ideal for service‑mesh scenarios; C++ codebase and higher learning curve for extensions.

For Java stacks, Spring Cloud Gateway is the preferred choice; for high‑performance or polyglot environments, APISIX or Envoy may be considered.

Production Practices

Large systems often run separate gateways for consumer (C‑side) and merchant (B‑side) traffic due to differing authentication, rate‑limiting and routing requirements.

Authentication Differences

C‑side uses JWT + SSO logout; B‑side adds guest tokens and RSA signature verification.

// B‑side guest token example
private static final TokenDetail FAKE_GUEST_TOKEN = new TokenDetail()
    .setSub("-1")
    .setChannel("weixin")
    .setIsGuest(true)
    .setSource("mini_program");
// B‑side RSA signature verification
String clientId = request.getClientId();
String payload = request.getPayload();
String sign = request.getSign();
if (Math.abs(nowTimestamp - timestamp) > signApiV2Properties.getTtl()) {
  throw new SignApiV2AccessNotAllowException("request timestamp too late or early");
}
SignUtils.checkSign(request, "sign", clientProperties.getPublicKey());

Rate‑Limiting Strategies

C‑side uses precise API token‑bucket limits; B‑side adds intelligent limiting that reacts to downstream circuit‑breakers.

Routing Management

C‑side routes are stored in Apollo configuration; B‑side routes reside in Redis with dynamic API updates and local snapshots for failover.

Capability : Auth – C‑side : JWT + SSO logout; B‑side : JWT + Guest mode + RSA signature

Rate limiting : C‑side – API precise limiting; B‑side – Precise + intelligent limiting

Circuit breaking : C‑side – Unified error response; B‑side – URL‑level circuit breaker + fallback

Routing : C‑side – Apollo config center; B‑side – Redis dynamic routing + local snapshot

Logging : Both – MQ async send

FAQ

What should a gateway do? Authentication, rate‑limiting, routing, logging, CORS, circuit breaking, request rewriting and response standardisation – all cross‑cutting concerns unrelated to business logic.

What should it not do? Business logic, data aggregation, direct database access, or complex decision‑making; mixing business code with the gateway compromises stability.

When to split gateways? When different client groups have distinct authentication methods, traffic patterns or security requirements, or when traffic isolation is needed to prevent one side’s spikes from affecting the other.

Is the gateway a single point of failure? Yes; typical mitigation is to place Nginx or LVS in front, deploy multiple gateway instances, and use health checks from the service registry. Keep the gateway lightweight and perform capacity testing.

Conclusion

The gateway acts like a security guard and front desk for a microservice system: it handles identity verification and traffic distribution, allowing downstream services to focus on their own responsibilities. Without a gateway, standards diverge, maintenance costs rise, security gaps appear, and operational risk increases. Building a gateway early avoids costly retrofits later, but the gateway itself should remain a thin, non‑business layer.