Mastering Gray Release with Spring Cloud Gateway: A Step‑by‑Step Guide

What is Gray Release

Gray release (also called canary release) is a deployment method that enables a smooth transition between versions, allowing A/B testing by routing a portion of users to feature A and the rest to feature B, gradually expanding if no objections arise, thus ensuring overall system stability.

Types of Gray Release

Canary Release

Diverts a small amount of traffic to the new version, requiring only a few machines. After validation, traffic weight is gradually increased, scaling up the new version and scaling down the old version to maximize resource utilization.

Advantages:

Proportionally routes traffic, limiting the impact of failures.

Allows simultaneous scaling of the new version and shrinking of the old version, achieving high resource utilization.

Disadvantages:

Indiscriminate traffic routing may affect important users.

Long release cycle.

A/B Testing

A/B testing routes traffic based on request metadata, such as headers or cookies. For example, requests whose User-Agent contains "Android" are sent to the new version, while others stay on the old version; or VIP users receive the new version while regular users remain on the old one.

Blue‑Green Deployment

Deploys a full duplicate of the service; the new version runs in parallel with the old one. Traffic is switched entirely to the new version when ready, and can be rolled back instantly by reverting traffic if serious bugs appear.

Implementing Gray Release with Spring Cloud Gateway

This article demonstrates implementing gray release via A/B testing in Spring Cloud Gateway.

Dependencies

<code><dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-gateway</artifactId>
  <version>3.1.4</version>
</dependency>
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-loadbalancer</artifactId>
  <version>3.1.4</version>
</dependency>
</code>

Custom Load Balancer

The custom load balancer selects service instances based on the "v" parameter in request headers or query parameters.

<code>public class GrayRoundRobinLoadBalancer implements ReactorServiceInstanceLoadBalancer {
    private static final Log log = LogFactory.getLog(RoundRobinLoadBalancer.class);
    final AtomicInteger position;
    final String serviceId;
    ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider;

    public GrayRoundRobinLoadBalancer(ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider,
                                     String serviceId) {
        this(serviceInstanceListSupplierProvider, serviceId, new Random().nextInt(1000));
    }

    public GrayRoundRobinLoadBalancer(ObjectProvider<ServiceInstanceListSupplier> serviceInstanceListSupplierProvider,
                                     String serviceId, int seedPosition) {
        this.serviceId = serviceId;
        this.serviceInstanceListSupplierProvider = serviceInstanceListSupplierProvider;
        this.position = new AtomicInteger(seedPosition);
    }

    @Override
    public Mono<Response<ServiceInstance>> choose(Request request) {
        ServiceInstanceListSupplier supplier = serviceInstanceListSupplierProvider
                .getIfAvailable(NoopServiceInstanceListSupplier::new);
        return supplier.get(request).next()
                .map(serviceInstances -> processInstanceResponse(supplier, serviceInstances, request));
    }

    private Response<ServiceInstance> processInstanceResponse(ServiceInstanceListSupplier supplier,
            List<ServiceInstance> serviceInstances, Request request) {
        Response<ServiceInstance> serviceInstanceResponse = getInstanceResponse(serviceInstances, request);
        if (supplier instanceof SelectedInstanceCallback && serviceInstanceResponse.hasServer()) {
            ((SelectedInstanceCallback) supplier).selectedServiceInstance(serviceInstanceResponse.getServer());
        }
        return serviceInstanceResponse;
    }

    private Response<ServiceInstance> getInstanceResponse(List<ServiceInstance> instances, Request request) {
        if (instances.isEmpty()) {
            if (log.isWarnEnabled()) {
                log.warn("No servers available for service: " + serviceId);
            }
            return new EmptyResponse();
        }
        List<ServiceInstance> result = instances.stream().filter(instance -> {
            Map<String, String> metadata = instance.getMetadata();
            Object orgId = metadata.get("v");
            RequestDataContext context = (RequestDataContext) request.getContext();
            RequestData requestData = context.getClientRequest();
            String v = null;
            if (requestData instanceof GrayRequestData) {
                GrayRequestData grayRequestData = (GrayRequestData) requestData;
                String queryV = grayRequestData.getQueryParams().getFirst("v");
                v = queryV;
            }
            String value = requestData.getHeaders().getFirst("v");
            return v != null && (v.equals(value) || v.equals(value));
        }).collect(Collectors.toList());
        if (result.isEmpty()) {
            result = instances;
        }
        int pos = this.position.incrementAndGet() & Integer.MAX_VALUE;
        ServiceInstance instance = result.get(pos % result.size());
        return new DefaultResponse(instance);
    }
}
</code>

Global Filter

The filter uses the custom load balancer to select a service instance for each request.

<code>@SuppressWarnings({ "rawtypes", "unchecked" })
@Component
public class GrayReactiveLoadBalancerClientFilter implements GlobalFilter, Ordered {
    public static final int LOAD_BALANCER_CLIENT_FILTER_ORDER = 10150;
    private final LoadBalancerClientFactory clientFactory;
    private final GatewayLoadBalancerProperties properties;

    public GrayReactiveLoadBalancerClientFilter(LoadBalancerClientFactory clientFactory,
                                                GatewayLoadBalancerProperties properties) {
        this.clientFactory = clientFactory;
        this.properties = properties;
    }

    @Override
    public int getOrder() {
        return LOAD_BALANCER_CLIENT_FILTER_ORDER;
    }

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        URI url = exchange.getAttribute(GATEWAY_REQUEST_URL_ATTR);
        String schemePrefix = exchange.getAttribute(GATEWAY_SCHEME_PREFIX_ATTR);
        if (url == null || (!"packlb".equals(url.getScheme()) && !"packlb".equals(schemePrefix))) {
            return chain.filter(exchange);
        }
        addOriginalRequestUrl(exchange, url);
        URI requestUri = exchange.getAttribute(GATEWAY_REQUEST_URL_ATTR);
        String serviceId = requestUri.getHost();
        Set<LoadBalancerLifecycle> supportedLifecycleProcessors = LoadBalancerLifecycleValidator
                .getSupportedLifecycleProcessors(clientFactory.getInstances(serviceId, LoadBalancerLifecycle.class),
                        RequestDataContext.class, ResponseData.class, ServiceInstance.class);
        DefaultRequest<RequestDataContext> lbRequest = new DefaultRequest<>(
                new RequestDataContext(new GrayRequestData(exchange.getRequest()), getHint(serviceId)));
        LoadBalancerProperties loadBalancerProperties = clientFactory.getProperties(serviceId);
        return choose(lbRequest, serviceId, supportedLifecycleProcessors).doOnNext(response -> {
            if (!response.hasServer()) {
                supportedLifecycleProcessors.forEach(lifecycle -> lifecycle
                        .onComplete(new CompletionContext<>(CompletionContext.Status.DISCARD, lbRequest, response)));
                throw NotFoundException.create(properties.isUse404(), "Unable to find instance for " + url.getHost());
            }
            ServiceInstance retrievedInstance = response.getServer();
            URI uri = exchange.getRequest().getURI();
            String overrideScheme = retrievedInstance.isSecure() ? "https" : "http";
            if (schemePrefix != null) {
                overrideScheme = url.getScheme();
            }
            DelegatingServiceInstance serviceInstance = new DelegatingServiceInstance(retrievedInstance, overrideScheme);
            URI requestUrl = reconstructURI(serviceInstance, uri);
            if (log.isTraceEnabled()) {
                log.trace("LoadBalancerClientFilter url chosen: " + requestUrl);
            }
            exchange.getAttributes().put(GATEWAY_REQUEST_URL_ATTR, requestUrl);
            exchange.getAttributes().put(GATEWAY_LOADBALANCER_RESPONSE_ATTR, response);
            supportedLifecycleProcessors.forEach(lifecycle -> lifecycle.onStartRequest(lbRequest, response));
        }).then(chain.filter(exchange))
                .doOnError(throwable -> supportedLifecycleProcessors.forEach(lifecycle -> lifecycle
                        .onComplete(new CompletionContext<>(CompletionContext.Status.FAILED, throwable, lbRequest,
                                exchange.getAttribute(GATEWAY_LOADBALANCER_RESPONSE_ATTR)))))
                .doOnSuccess(aVoid -> supportedLifecycleProcessors.forEach(lifecycle -> lifecycle
                        .onComplete(new CompletionContext<>(CompletionContext.Status.SUCCESS, lbRequest,
                                exchange.getAttribute(GATEWAY_LOADBALANCER_RESPONSE_ATTR),
                                buildResponseData(exchange, loadBalancerProperties.isUseRawStatusCodeInResponseData())))));
    }

    private ResponseData buildResponseData(ServerWebExchange exchange, boolean useRawStatusCodes) {
        if (useRawStatusCodes) {
            return new ResponseData(new GrayRequestData(exchange.getRequest()), exchange.getResponse());
        }
        return new ResponseData(exchange.getResponse(), new RequestData(exchange.getRequest()));
    }

    protected URI reconstructURI(ServiceInstance serviceInstance, URI original) {
        return LoadBalancerUriTools.reconstructURI(serviceInstance, original);
    }

    private Mono<Response<ServiceInstance>> choose(Request<RequestDataContext> lbRequest, String serviceId,
            Set<LoadBalancerLifecycle> supportedLifecycleProcessors) {
        ReactorLoadBalancer<ServiceInstance> loadBalancer = this.clientFactory.getInstance(serviceId,
                ReactorServiceInstanceLoadBalancer.class);
        if (loadBalancer == null) {
            throw new NotFoundException("No loadbalancer available for " + serviceId);
        }
        supportedLifecycleProcessors.forEach(lifecycle -> lifecycle.onStart(lbRequest));
        return loadBalancer.choose(lbRequest);
    }

    private String getHint(String serviceId) {
        LoadBalancerProperties loadBalancerProperties = clientFactory.getProperties(serviceId);
        Map<String, String> hints = loadBalancerProperties.getHint();
        String defaultHint = hints.getOrDefault("default", "default");
        String hintPropertyValue = hints.get(serviceId);
        return hintPropertyValue != null ? hintPropertyValue : defaultHint;
    }
}
</code>

Configuration

<code>@Configuration
public class GrayDefaultConfiguration {

    @Bean
    public GrayRoundRobinLoadBalancer grayRandomLoadBalancer(Environment environment,
                                                             LoadBalancerClientFactory loadBalancerClientFactory) {
        String name = environment.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
        return new GrayRoundRobinLoadBalancer(
                loadBalancerClientFactory.getLazyProvider(name, ServiceInstanceListSupplier.class), name);
    }

    // Define service instances manually because service registry is not used
    @Bean
    public ServiceInstanceListSupplier sscServiceInstanceListSupplier() {
        return new ServiceInstanceListSupplier() {
            @Override
            public Flux<List<ServiceInstance>> get() {
                List<ServiceInstance> instances = new ArrayList<>();
                Map<String, String> metadata1 = new HashMap<>();
                metadata1.put("v", "1");
                ServiceInstance s1 = new DefaultServiceInstance("s1", "ssc", "localhost", 8088, false, metadata1);
                instances.add(s1);

                Map<String, String> metadata2 = new HashMap<>();
                metadata2.put("v", "2");
                ServiceInstance s2 = new DefaultServiceInstance("s2", "ssc", "localhost", 8099, false, metadata2);
                instances.add(s2);

                return Flux.just(instances);
            }

            @Override
            public String getServiceId() {
                return "ssc";
            }
        };
    }
}
</code>

Finally, set the custom configuration as the default load‑balancer client configuration:

<code>@LoadBalancerClients(defaultConfiguration = GrayDefaultConfiguration.class)
public class SpringCloudGatewayApplication {
}
</code>

These components constitute the core implementation of gray release in a Spring Cloud Gateway environment.