Implementing End-to-End Gray Release with Spring Cloud, Nacos, and Load Balancer

Gray release, also known as canary deployment, gradually shifts traffic from an old version to a new version, allowing early detection of issues. The article demonstrates a full‑stack gray release solution built on Spring Cloud, Nacos, and a load balancer.

Component Versions

spring-boot: 2.3.12.RELEASE

spring-cloud-dependencies: Hoxton.SR12

spring-cloud-alibaba-dependencies: 2.2.9.RELEASE

Core Components

Registration center: Nacos

Gateway: Spring Cloud Gateway

Load balancer: Ribbon (or Spring Cloud LoadBalancer)

RPC: OpenFeign

Gray Flag Holder

public class GrayFlagRequestHolder {
    private static final ThreadLocal<GrayStatusEnum> grayFlag = new ThreadLocal<>();
    public static void setGrayTag(final GrayStatusEnum tag) { grayFlag.set(tag); }
    public static GrayStatusEnum getGrayTag() { return grayFlag.get(); }
    public static void remove() { grayFlag.remove(); }
}

Gateway Filters

Pre‑filter determines whether the request should be routed to a gray version and stores the result in GrayFlagRequestHolder. It sets the filter order to Ordered.HIGHEST_PRECEDENCE so it runs first.

public class GrayGatewayBeginFilter implements GlobalFilter, Ordered {
    @Autowired
    private GrayGatewayProperties grayGatewayProperties;
    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        GrayStatusEnum grayStatusEnum = GrayStatusEnum.ALL;
        if (grayGatewayProperties.getEnabled()) {
            grayStatusEnum = GrayStatusEnum.PROD;
            if (checkGray(exchange.getRequest())) {
                grayStatusEnum = GrayStatusEnum.GRAY;
            }
        }
        GrayFlagRequestHolder.setGrayTag(grayStatusEnum);
        ServerHttpRequest newRequest = exchange.getRequest().mutate()
                .header(GrayConstant.GRAY_HEADER, grayStatusEnum.getVal())
                .build();
        ServerWebExchange newExchange = exchange.mutate().request(newRequest).build();
        return chain.filter(newExchange);
    }
    private boolean checkGray(ServerHttpRequest request) {
        return checkGrayHeadKey(request) || checkGrayIPList(request) ||
               checkGrayCiryList(request) || checkGrayUserNoList(request);
    }
    // ... implementations of checkGrayHeadKey, checkGrayIPList, checkGrayCiryList, checkGrayUserNoList ...
    @Override
    public int getOrder() { return Ordered.HIGHEST_PRECEDENCE; }
}

Post‑filter clears the ThreadLocal to avoid memory leaks.

public class GrayGatewayAfterFilter implements GlobalFilter, Ordered {
    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        GrayFlagRequestHolder.remove();
        return chain.filter(exchange);
    }
    @Override
    public int getOrder() { return Ordered.LOWEST_PRECEDENCE; }
}

Global Exception Handler

public class GrayGatewayExceptionHandler implements WebExceptionHandler, Ordered {
    @Override
    public Mono<Void> handle(ServerWebExchange exchange, Throwable ex) {
        GrayFlagRequestHolder.remove();
        ServerHttpResponse response = exchange.getResponse();
        if (ex instanceof ResponseStatusException) {
            response.setStatusCode(((ResponseStatusException) ex).getStatus());
        } else {
            response.setStatusCode(HttpStatus.INTERNAL_SERVER_ERROR);
        }
        return response.setComplete();
    }
    @Override
    public int getOrder() { return Ordered.HIGHEST_PRECEDENCE; }
}

Custom Ribbon Load‑Balancing Rule

The rule reads the gray flag from GrayFlagRequestHolder and selects the appropriate service list based on the version metadata stored in Nacos.

public abstract class AbstractGrayLoadBalancerRule extends AbstractLoadBalancerRule {
    @Autowired
    private GrayVersionProperties grayVersionProperties;
    @Value("${spring.cloud.nacos.discovery.metadata.version}")
    private String metaVersion;
    public List<Server> getReachableServers() {
        ILoadBalancer lb = getLoadBalancer();
        if (lb == null) return new ArrayList<>();
        return getGrayServers(lb.getReachableServers());
    }
    public List<Server> getAllServers() {
        ILoadBalancer lb = getLoadBalancer();
        if (lb == null) return new ArrayList<>();
        return getGrayServers(lb.getAllServers());
    }
    protected List<Server> getGrayServers(List<Server> servers) {
        List<Server> result = new ArrayList<>();
        if (servers == null) return result;
        String currentVersion = metaVersion;
        GrayStatusEnum grayStatusEnum = GrayFlagRequestHolder.getGrayTag();
        if (grayStatusEnum != null) {
            switch (grayStatusEnum) {
                case PROD: currentVersion = grayVersionProperties.getProdVersion(); break;
                case GRAY: currentVersion = grayVersionProperties.getGrayVersion(); break;
                default: return servers; // ALL
            }
        }
        for (Server server : servers) {
            NacosServer nacosServer = (NacosServer) server;
            String version = nacosServer.getMetadata().get("version");
            if (version != null && version.equals(currentVersion)) {
                result.add(server);
            }
        }
        return result;
    }
}

Custom Spring MVC Interceptor

public class GrayMvcHandlerInterceptor implements HandlerInterceptor {
    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
        String grayTag = request.getHeader(GrayConstant.GRAY_HEADER);
        if (grayTag != null) {
            GrayFlagRequestHolder.setGrayTag(GrayStatusEnum.getByVal(grayTag));
        }
        return true;
    }
    @Override
    public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) {
        GrayFlagRequestHolder.remove();
    }
}

Custom OpenFeign Interceptor

public class GrayFeignRequestInterceptor implements RequestInterceptor {
    @Override
    public void apply(RequestTemplate template) {
        GrayStatusEnum grayStatusEnum = GrayFlagRequestHolder.getGrayTag();
        if (grayStatusEnum != null) {
            template.header(GrayConstant.GRAY_HEADER, Collections.singleton(grayStatusEnum.getVal()));
        }
    }
}

Configuration Files

common-config.yaml (Nacos global config)

kerwin:
  tool:
    gray:
      load: true
      version:
        prodVersion: V1
        grayVersion: V2
    user-app:
      ribbon:
        NFLoadBalancerRuleClassName: com.kerwin.gray.loadbalancer.GrayRoundRobinRule
    order-app:
      ribbon:
        NFLoadBalancerRuleClassName: com.kerwin.gray.loadbalancer.GrayRoundRobinRule

gateway-app.yaml (Gateway Nacos config)

kerwin:
  tool:
    gray:
      gateway:
        enabled: true
        grayHeadKey: gray
        grayHeadValue: gray-996
        grayIPList:
          - '127.0.0.1'
        grayCityList:
          - 本地

Running the Demo

The demo starts five services: a gateway, user‑service V1, order‑service V1, user‑service V2, and order‑service V2. Each service is launched with JVM arguments that set the port and the Nacos metadata version (e.g.,

-Dserver.port=7201 -Dspring.cloud.nacos.discovery.metadata.version=V1

).

Gray Release Scenarios

Scenario 1 – Gray switch off : Both kerwin.tool.gray.load and kerwin.tool.gray.gateway.enabled are false, so all requests go to whichever service instance is available, without version filtering.

Scenario 2 – Gray switch on, only production version : gateway.enabled is true while IP, city, and header lists do not match, so the system always selects the production version (V1).

Scenario 3 – Gray switch on, header/IP/city match : Supplying the header gray=gray-996 (or matching IP/city) causes the request to be routed to the gray version (V2).

Open Issues

Distributed task schedulers (e.g., XXL‑Job) need separate executors per version.

Message queues require separate instances per version to keep traffic isolated.

The presented approach is one possible solution; alternatives such as Nginx + Lua routing with separate Nacos namespaces can also achieve isolation.