Implementing Gray Release with Spring Cloud Gateway

Deployment vs Release

Deployment copies the application and its configuration to target machines and starts it. Release is the moment the service begins handling live traffic. Separating these stages enables safer rollouts.

Common Release Strategies

Rolling Update : Deploy instances in small batches, verify each batch, and roll back on failure. Traffic proportion follows the number of instances (M/N).

Gray (Canary) Release : Incrementally increase the proportion of traffic sent to the new version (e.g., 10 % → 20 % → 50 % → 100 %).

Blue‑Green Deployment : Run two complete environments in parallel and switch all traffic once the new environment is verified.

Gray Release via Spring Cloud Gateway Weight Route Predicate

Spring Cloud Gateway provides a Weight Route Predicate that splits traffic by percentage. Example YAML for two versions each receiving 50 % of the traffic:

spring:
  cloud:
    gateway:
      routes:
      - id: app-server-a
        uri: lb://app-server-a
        predicates:
        - Path=/app-server/**
        - Weight=app-server,50
        filters:
        - RewritePath=/app-server(?<segment>/?.*,${segment})
      - id: app-server-b
        uri: lb://app-server-b
        predicates:
        - Path=/app-server/**
        - Weight=app-server,50
        filters:
        - RewritePath=/app-server(?<segment>/?.*,${segment})

The weight values can be changed at runtime through a configuration center (e.g., Apollo or Spring Cloud Config) to adjust the split.

Weighted Service Instance List Supplier

Configure each service instance with metadata-map.weight (e.g., 80 for version A, 20 for version B) and set the load‑balancer strategy to weighted. The gateway then distributes requests proportionally.

eureka:
  instance:
    metadata-map:
      weight: 80

spring:
  cloud:
    loadbalancer:
      clients:
        app-server:
          configurations: weighted

Hint‑Based Service Instance List Supplier

The hintBasedServiceInstanceListSupplier examines the request header X‑SC‑LB‑Hint and routes the request to instances whose metadata-map.hint matches the header value. This is useful when new APIs exist only in the new version.

spring:
  cloud:
    gateway:
      routes:
      - id: app-server-v2
        uri: lb://app-server
        predicates:
        - Path=/app-server/v2/**
        filters:
        - RewritePath=/app-server(?<segment>/?.*,${segment})
        - AddRequestHeader=X-SC-LB-Hint,v2
      - id: app-server-v1
        uri: lb://app-server
        predicates:
        - Path=/app-server/**
        filters:
        - RewritePath=/app-server(?<segment>/?.*,${segment})

Service instances declare their version hint:

eureka:
  instance:
    metadata-map:
      hint: v2

Custom ServiceInstanceListSupplier

A custom supplier combines weight‑based and hint‑based routing. The class WeightedVersionServiceInstanceListSupplier parses a version-weight configuration (e.g., v1,80,v2,20), generates a random number, selects a version according to the configured ratios, and filters instances by hint or weight. Core implementation:

public class WeightedVersionServiceInstanceListSupplier extends DelegatingServiceInstanceListSupplier {
    private static ThreadLocal<VersionWeightInfo> versionWeightInfoThreadLocal = new ThreadLocal<>();
    private final ReactiveLoadBalancer.Factory<ServiceInstance> factory;
    private final Splitter splitter = Splitter.on(",").omitEmptyStrings().trimResults();

    public WeightedVersionServiceInstanceListSupplier(ServiceInstanceListSupplier delegate,
            ReactiveLoadBalancer.Factory<ServiceInstance> factory) {
        super(delegate);
        this.factory = factory;
    }

    @Override
    public Flux<List<ServiceInstance>> get(Request request) {
        return delegate.get(request)
                .map(instances -> filteredByHintAndWeightedVersion(instances, getHint(request.getContext())));
    }

    private void parseWeightedVersionConfig() {
        VersionWeightInfo info = versionWeightInfoThreadLocal.get();
        LoadBalancerProperties props = factory.getProperties(getServiceId());
        String versionWeights = props.getHint().getOrDefault("version-weight", "");
        if (versionWeights == null || versionWeights.isEmpty()) {
            versionWeightInfoThreadLocal.set(new VersionWeightInfo());
            return;
        }
        // parse "v1,80,v2,20" into a map and compute normalized ranges (omitted)
    }

    private String getHint(Object requestContext) { /* extract X‑SC‑LB‑Hint header */ }
    private List<ServiceInstance> filteredByHint(List<ServiceInstance> instances, String hint) { /* filter by metadata hint */ }
    private List<ServiceInstance> filteredByWeightedVersion(List<ServiceInstance> instances) { /* select version by random weight */ }
    private List<ServiceInstance> filteredByHintAndWeightedVersion(List<ServiceInstance> instances, String hint) {
        List<ServiceInstance> filtered = filteredByHint(instances, hint);
        if (!filtered.isEmpty()) return filtered;
        filtered = filteredByWeightedVersion(instances);
        return filtered.isEmpty() ? instances : filtered;
    }

    private static class VersionWeightInfo {
        private final String versionWeights;
        private final boolean validVersionWeight;
        private final LinkedHashMap<Integer, String> rangeIndexes;
        private final List<Double> ranges;
        private final Random random = new Random();

        public VersionWeightInfo() {
            this.versionWeights = "";
            this.validVersionWeight = false;
            this.rangeIndexes = new LinkedHashMap<>();
            this.ranges = new ArrayList<>();
        }

        public VersionWeightInfo(String versionWeights, boolean validVersionWeight,
                                 LinkedHashMap<Integer, String> rangeIndexes, List<Double> ranges) {
            this.versionWeights = versionWeights;
            this.validVersionWeight = validVersionWeight;
            this.rangeIndexes = rangeIndexes;
            this.ranges = ranges;
        }

        public String getVersionWeights() { return versionWeights; }
        public boolean getValidVersionWeight() { return validVersionWeight; }
        public LinkedHashMap<Integer, String> getRangeIndexes() { return rangeIndexes; }
        public List<Double> getRanges() { return ranges; }
        public Random getRandom() { return random; }
    }
}

Register the custom supplier with the load balancer:

public class AppLoadBalancerClientConfiguration {
    @Bean
    public ServiceInstanceListSupplier weightedVersionServiceInstanceListSupplier(ConfigurableApplicationContext context) {
        DelegateCreator creator = (ctx, delegate) -> {
            LoadBalancerClientFactory factory = ctx.getBean(LoadBalancerClientFactory.class);
            return new WeightedVersionServiceInstanceListSupplier(delegate, factory);
        };
        return ServiceInstanceListSupplier.builder()
                .withBlockingDiscoveryClient()
                .with(creator)
                .withCaching()
                .build(context);
    }
}

@LoadBalancerClients(defaultConfiguration = AppLoadBalancerClientConfiguration.class)
@SpringBootApplication
public class AppClient1Application {
    public static void main(String[] args) {
        SpringApplication.run(AppClient1Application.class, args);
    }
}

Configure the traffic split via the load‑balancer hint property:

spring:
  cloud:
    loadbalancer:
      clients:
        app-server:
          hint:
            version-weight: v1,80,v2,20

This setup routes new‑API calls (identified by the v2 path prefix) exclusively to version 2 instances, while unchanged APIs continue to be split according to the configured percentages until the old version can be retired.