Implementing Gray Release in Spring Cloud Using Nacos and Load Balancer

Concept

Gray (canary) release routes a subset of traffic to a new version and expands it after verification, ensuring system stability while allowing incremental migration. AB testing is a typical gray‑release pattern.

Component Versions

spring-boot: 2.3.12.RELEASE

spring-cloud-dependencies: Hoxton.SR12

spring-cloud-alibaba-dependencies: 2.2.9.RELEASE

Core Components

Registration Center: Nacos

Gateway: Spring Cloud Gateway

Load Balancer: Ribbon (or Spring Cloud LoadBalancer)

RPC Call: OpenFeign

Gray Flag Holder

A ThreadLocal stores the gray status for each request. Downstream components read and clear this value to avoid memory leaks.

public class GrayFlagRequestHolder {
    private static final ThreadLocal<GrayStatusEnum> grayFlag = new ThreadLocal<>();
    public static void setGrayTag(final GrayStatusEnum tag) { grayFlag.set(tag); }
    public static GrayStatusEnum getGrayTag() { return grayFlag.get(); }
    public static void remove() { grayFlag.remove(); }
}

Gateway Pre‑Filter

The pre‑filter decides whether a request should be treated as gray based on a switch, request header, IP list, city list, or user list. It writes the selected GrayStatusEnum into the request header and stores it in GrayFlagRequestHolder.

public class GrayGatewayBeginFilter implements GlobalFilter, Ordered {
    @Autowired private GrayGatewayProperties grayGatewayProperties;
    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        GrayStatusEnum grayStatusEnum = GrayStatusEnum.ALL;
        if (grayGatewayProperties.getEnabled()) {
            grayStatusEnum = GrayStatusEnum.PROD;
            if (checkGray(exchange.getRequest())) {
                grayStatusEnum = GrayStatusEnum.GRAY;
            }
        }
        GrayFlagRequestHolder.setGrayTag(grayStatusEnum);
        ServerHttpRequest newRequest = exchange.getRequest().mutate()
                .header(GrayConstant.GRAY_HEADER, grayStatusEnum.getVal())
                .build();
        ServerWebExchange newExchange = exchange.mutate().request(newRequest).build();
        return chain.filter(newExchange);
    }
    // checkGray, checkGrayHeadKey, checkGrayIPList, checkGrayCiryList, checkGrayUserNoList ...
    @Override public int getOrder() { return Ordered.HIGHEST_PRECEDENCE; }
}

Gateway Post‑Filter

After the downstream call finishes, the post‑filter clears the ThreadLocal to prevent memory leaks.

public class GrayGatewayAfterFilter implements GlobalFilter, Ordered {
    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        GrayFlagRequestHolder.remove();
        return chain.filter(exchange);
    }
    @Override public int getOrder() { return Ordered.LOWEST_PRECEDENCE; }
}

Global Exception Handler

If an exception occurs before the post‑filter runs, the handler also removes the holder and sets an appropriate HTTP status.

public class GrayGatewayExceptionHandler implements WebExceptionHandler, Ordered {
    @Override
    public Mono<Void> handle(ServerWebExchange exchange, Throwable ex) {
        GrayFlagRequestHolder.remove();
        ServerHttpResponse response = exchange.getResponse();
        if (ex instanceof ResponseStatusException) {
            response.setStatusCode(((ResponseStatusException) ex).getStatus());
        } else {
            response.setStatusCode(HttpStatus.INTERNAL_SERVER_ERROR);
        }
        return response.setComplete();
    }
    @Override public int getOrder() { return Ordered.HIGHEST_PRECEDENCE; }
}

Custom Ribbon Load‑Balancing

An abstract rule filters the service list according to the gray status stored in GrayFlagRequestHolder. It returns all servers for ALL, only production‑version servers for PROD, and only gray‑version servers for GRAY.

public abstract class AbstractGrayLoadBalancerRule extends AbstractLoadBalancerRule {
    @Autowired private GrayVersionProperties grayVersionProperties;
    @Value("${spring.cloud.nacos.discovery.metadata.version}") private String metaVersion;
    @Override
    public List<Server> getReachableServers() {
        ILoadBalancer lb = getLoadBalancer();
        if (lb == null) return new ArrayList<>();
        List<Server> reachable = lb.getReachableServers();
        return getGrayServers(reachable);
    }
    protected List<Server> getGrayServers(List<Server> servers) {
        List<Server> result = new ArrayList<>();
        if (servers == null) return result;
        String currentVersion = metaVersion;
        GrayStatusEnum status = GrayFlagRequestHolder.getGrayTag();
        if (status != null) {
            switch (status) {
                case ALL: return servers;
                case PROD: currentVersion = grayVersionProperties.getProdVersion(); break;
                case GRAY: currentVersion = grayVersionProperties.getGrayVersion(); break;
            }
        }
        for (Server server : servers) {
            NacosServer ns = (NacosServer) server;
            String version = ns.getMetadata().get("version");
            if (version != null && version.equals(currentVersion)) {
                result.add(server);
            }
        }
        return result;
    }
}

A concrete GrayRoundRobinRule extends this abstract class and reuses Ribbon’s round‑robin algorithm.

Business Service Gray Design

Spring MVC Interceptor

The interceptor copies the gray header from the upstream request into GrayFlagRequestHolder so that downstream Feign calls inherit the flag.

@SuppressWarnings("all")
public class GrayMvcHandlerInterceptor implements HandlerInterceptor {
    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
        String grayTag = request.getHeader(GrayConstant.GRAY_HEADER);
        if (grayTag != null) {
            GrayFlagRequestHolder.setGrayTag(GrayStatusEnum.getByVal(grayTag));
        }
        return true;
    }
    @Override public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) {
        GrayFlagRequestHolder.remove();
    }
}

OpenFeign Interceptor

The Feign interceptor reads the flag from the holder and adds the gray header to outbound Feign requests.

public class GrayFeignRequestInterceptor implements RequestInterceptor {
    @Override
    public void apply(RequestTemplate template) {
        GrayStatusEnum grayStatusEnum = GrayFlagRequestHolder.getGrayTag();
        if (grayStatusEnum != null) {
            template.header(GrayConstant.GRAY_HEADER, Collections.singleton(grayStatusEnum.getVal()));
        }
    }
}

Basic Configuration Classes

public interface GrayConstant { String GRAY_HEADER = "gray"; }

public enum GrayStatusEnum {
    ALL("ALL","can call all versions"),
    PROD("PROD","only production version"),
    GRAY("GRAY","only gray version");
    private final String val; private final String desc;
    GrayStatusEnum(String val, String desc) { this.val = val; this.desc = desc; }
    public String getVal() { return val; }
    public static GrayStatusEnum getByVal(String val) {
        if (val == null) return null;
        for (GrayStatusEnum e : values()) {
            if (e.val.equals(val)) return e;
        }
        return null;
    }
}

@Data @Configuration @RefreshScope @ConfigurationProperties("kerwin.tool.gray.gateway")
public class GrayGatewayProperties {
    private Boolean enabled = false;
    private String grayHeadKey = "gray";
    private String grayHeadValue = "gray-996";
    private List<String> grayIPList = new ArrayList<>();
    private List<String> grayCityList = new ArrayList<>();
    private List<String> grayUserNoList = new ArrayList<>();
}

@Data @Configuration @RefreshScope @ConfigurationProperties("kerwin.tool.gray.version")
public class GrayVersionProperties {
    private String prodVersion;
    private String grayVersion;
}

Auto‑Configuration

Conditional beans create the gateway filters, MVC interceptor, and Feign interceptor only when the corresponding classes are present and the property kerwin.tool.gray.load=true is set.

@Configuration
@ConditionalOnProperty(value = "kerwin.tool.gray.load", havingValue = "true")
@EnableConfigurationProperties(GrayVersionProperties.class)
public class GrayAutoConfiguration {
    @Configuration(proxyBeanMethods = false)
    @ConditionalOnClass(GlobalFilter.class)
    @EnableConfigurationProperties(GrayGatewayProperties.class)
    static class GrayGatewayFilterAutoConfiguration {
        @Bean public GrayGatewayBeginFilter grayGatewayBeginFilter() { return new GrayGatewayBeginFilter(); }
        @Bean public GrayGatewayAfterFilter grayGatewayAfterFilter() { return new GrayGatewayAfterFilter(); }
        @Bean public GrayGatewayExceptionHandler grayGatewayExceptionHandler() { return new GrayGatewayExceptionHandler(); }
    }
    @Configuration(proxyBeanMethods = false)
    @ConditionalOnClass(WebMvcConfigurer.class)
    static class GrayWebMvcAutoConfiguration {
        @Bean public WebMvcConfigurer webMvcConfigurer() {
            return new WebMvcConfigurer() {
                @Override public void addInterceptors(InterceptorRegistry registry) {
                    registry.addInterceptor(new GrayMvcHandlerInterceptor());
                }
            };
        }
    }
    @Configuration
    @ConditionalOnClass(RequestInterceptor.class)
    static class GrayFeignInterceptorAutoConfiguration {
        @Bean public GrayFeignRequestInterceptor grayFeignRequestInterceptor() { return new GrayFeignRequestInterceptor(); }
    }
}

Project Structure

spring-cloud-gray-example // parent project
  ├─ kerwin-common // common module
  ├─ kerwin-gateway // gateway service
  ├─ kerwin-order
  │   └─ order-app // order business service
  ├─ kerwin-starter
  │   └─ spring-cloud-starter-kerwin-gray // core gray code
  └─ kerwin-user
      └─ user-app // user business service

Running the Demo

Five services are started: one gateway, user‑v1, order‑v1, user‑v2, order‑v2. Nacos global configuration defines the production version V1 and gray version V2. Each service registers its version in metadata (e.g., -Dspring.cloud.nacos.discovery.metadata.version=V1 for production).

Gateway configuration ( gateway‑app.yaml) enables gray, sets the header key/value, and optionally lists IPs or cities for gray routing.

Gray Effect Demonstration

Scenario 1 – Gray switch off: All requests may hit any version because the gateway does not evaluate gray rules.

Scenario 2 – Gray switch on, only production: With no matching IP/header, the gateway forces PROD and routes to V1 services.

Scenario 3 – Gray switch on, header match: Sending gray=gray-996 forces GRAY and routes to V2 services.

Source Code

https://gitee.com/kerwin_code/spring-cloud-gray-example

Open Issues & Suggested Solutions

Distributed task scheduling (e.g., XXL‑Job): Register separate executors for gray and production versions; gray deployments use the gray executor.

Message queues: Deploy separate MQ clusters for gray and production, routing messages from gray services to the gray MQ cluster.

Complexity reduction: Isolate gray services in a dedicated Nacos namespace and let Nginx + Lua perform routing, keeping configuration independent from the Spring Cloud stack.