Mastering API Gateways with Spring Cloud Gateway: Routing, Rate Limiting, and Dynamic Configuration

Why API Gateways are needed

In microservice architectures a single client request often has to invoke multiple services (e.g., product lookup, inventory deduction, order update). Directly calling each service increases client complexity and network overhead. An API gateway provides a single entry point that aggregates these calls, reduces latency, and hides the internal service topology.

Spring Cloud Gateway fundamentals

Spring Cloud Gateway is a cloud‑native API‑gateway implementation. Its core concepts are:

Route : maps an incoming request to a target URI. A route has a unique ID, a list of predicates, and a list of filters.

Predicate : evaluates request attributes (path, method, headers, etc.) to decide whether the route matches. It works like a series of if checks.

Filter : intercepts the request/response for pre‑processing or post‑processing. Filters can be scoped to a single route (Gateway Filter) or applied globally (Global Filter).

Spring Cloud Gateway supports two filter scopes:

Gateway Filter – applied to a specific route or route group.

Global Filter – applied to all routes.

Request processing flow

When a client sends a request, HttpWebHandlerAdapter creates a gateway context and forwards it to DispatcherHandler. The dispatcher uses RoutePredicateHandlerMapping to locate the matching route, builds a filter chain, and then forwards the request to the downstream microservice. Filters run both before (pre‑filter) and after (post‑filter) the downstream service processes the request.

Basic routing

Add the starter dependency to pom.xml:

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>

Java configuration (programmatic):

@Bean
public RouteLocator routeLocator(RouteLocatorBuilder builder) {
    return builder.routes()
        .route(r -> r.path("/baidu")
            .uri("http://www.baidu.com/")
            .id("baidu_route"))
        .build();
}

YAML configuration (equivalent):

spring:
  cloud:
    gateway:
      routes:
        - id: baidu_route
          uri: http://baidu.com:80/
          predicates:
            - Path=/baidu

Weighted (gray) routing

Weight filters allow gradual traffic shift between service versions. Example application.yml routes 90 % of traffic to service_old and 10 % to service_new:

server:
  port: 8080
spring:
  cloud:
    gateway:
      routes:
        - id: service_old
          uri: http://localhost:8888/v1
          predicates:
            - Path=/gatewaytest
          filters:
            - Weight=service,90
        - id: service_new
          uri: http://localhost:8888/v2
          predicates:
            - Path=/gatewaytest
          filters:
            - Weight=service,10

Rate limiting with Bucket4j

Implement a token‑bucket algorithm to protect services from overload. Add the Bucket4j dependency:

<dependency>
  <groupId>com.github.vladimir-bukhtoyarov</groupId>
  <artifactId>bucket4j-core</artifactId>
  <version>4.0.0</version>
</dependency>

Custom filter example (IP‑based token bucket):

public class GatewayRateLimitFilterByIp implements GatewayFilter, Ordered {
    private static final Map<String, Bucket> LOCAL_CACHE = new ConcurrentHashMap<>();
    private final int capacity;
    private final int refillTokens;
    private final Duration refillDuration;

    public GatewayRateLimitFilterByIp(int capacity, int refillTokens, Duration refillDuration) {
        this.capacity = capacity;
        this.refillTokens = refillTokens;
        this.refillDuration = refillDuration;
    }

    private Bucket createNewBucket() {
        Refill refill = Refill.of(refillTokens, refillDuration);
        Bandwidth limit = Bandwidth.classic(capacity, refill);
        return Bucket4j.builder().addLimit(limit).build();
    }

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        String ip = exchange.getRequest().getRemoteAddress().getAddress().getHostAddress();
        Bucket bucket = LOCAL_CACHE.computeIfAbsent(ip, k -> createNewBucket());
        if (bucket.tryConsume(1)) {
            return chain.filter(exchange);
        } else {
            exchange.getResponse().setStatusCode(HttpStatus.TOO_MANY_REQUESTS);
            return exchange.getResponse().setComplete();
        }
    }

    @Override
    public int getOrder() { return 0; }
}

Register the filter in a route definition:

@Bean
public RouteLocator testRouteLocator(RouteLocatorBuilder builder) {
    return builder.routes()
        .route(r -> r.path("/rateLimit")
            .filters(f -> f.filter(new GatewayRateLimitFilterByIp(20, 1, Duration.ofSeconds(2))))
            .uri("http://localhost:8888/rateLimit")
            .id("rateLimit_route"))
        .build();
}

Dynamic routing without restart

Routes are static after the gateway starts. To modify them at runtime, expose an API that uses RouteDefinitionWriter and publishes a RefreshRoutesEvent after each change.

@Service
public class RouteServiceImpl implements ApplicationEventPublisherAware {
    @Autowired
    private RouteDefinitionWriter routeDefinitionWriter;
    private ApplicationEventPublisher publisher;

    public String add(RouteDefinition definition) {
        routeDefinitionWriter.save(Mono.just(definition)).subscribe();
        publisher.publishEvent(new RefreshRoutesEvent(this));
        return "success";
    }

    public String update(RouteDefinition definition) {
        try {
            routeDefinitionWriter.delete(Mono.just(definition.getId()));
            routeDefinitionWriter.save(Mono.just(definition)).subscribe();
            publisher.publishEvent(new RefreshRoutesEvent(this));
            return "success";
        } catch (Exception e) {
            return "failure";
        }
    }

    public String delete(String id) {
        try {
            routeDefinitionWriter.delete(Mono.just(id));
            publisher.publishEvent(new RefreshRoutesEvent(this));
            return "delete success";
        } catch (Exception e) {
            return "failure";
        }
    }

    @Override
    public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher) {
        this.publisher = applicationEventPublisher;
    }
}

Controller that forwards JSON payloads to the service:

@RestController
public class RouteController {
    @Autowired
    private RouteServiceImpl routeService;

    @PostMapping("/add")
    public String add(@RequestBody RouteDefinition rd) {
        return routeService.add(rd);
    }

    @PostMapping("/update")
    public String update(@RequestBody RouteDefinition rd) {
        return routeService.update(rd);
    }

    @DeleteMapping("/delete/{id}")
    public String delete(@PathVariable String id) {
        return routeService.delete(id);
    }
}

Example JSON to add a route that forwards /baidu to https://www.baidu.com:

{
  "id":"baidu_route",
  "uri":"https://www.baidu.com",
  "predicates":[{"name":"Path","args":{"pattern":"/baidu"}}],
  "filters":[]
}

POST the JSON to http://localhost:8888/route/add. The gateway immediately routes /baidu to Baidu without a restart. Updating or deleting routes follows the same pattern.

Conclusion

API gateways are essential in microservice ecosystems for request aggregation, routing, load balancing, rate limiting, caching, and logging. Spring Cloud Gateway implements these functions with clear concepts—Route, Predicate, and Filter—and supports practical scenarios such as basic routing, weighted (gray) releases, token‑bucket rate limiting, and dynamic route management without service downtime.