How to Build a Custom Spring Cloud Starter for Dynamic Gray Release

Why a Custom Gray LoadBalancer Is Needed

In microservice environments, the simple round‑robin or random load‑balancing strategies cannot satisfy scenarios such as canary releases, where only a small portion of real traffic should be routed to a new version for stability verification. A custom component must understand service metadata (version, region, etc.) and make routing decisions based on configurable rules.

Spring Cloud LoadBalancer Internals

Since Spring Cloud 2020.0.0 (Ilford), Netflix Ribbon was removed and Spring Cloud LoadBalancer became the official client‑side load‑balancing solution. Its architecture consists of three core components:

ReactorServiceInstanceLoadBalancer : implements the actual load‑balancing algorithm and selects a ServiceInstance for a request.

ServiceInstanceListSupplier : queries the service registry (e.g., Nacos) and supplies a list of healthy instances.

ServiceInstance : represents a concrete service node with IP, port, and metadata.

The key interface is:

public interface ReactorServiceInstanceLoadBalancer extends ReactorLoadBalancer<ServiceInstance> {
    Mono<Response<ServiceInstance>> choose(Request request);
}

The default implementation RoundRobinLoadBalancer cycles through instances using an atomic counter.

Custom Local‑First Strategy

Before tackling gray release, a simple "local‑first" strategy is introduced. It prefers instances whose IP matches the local machine; if none are found, it falls back to a random choice. The core method is:

private Response<ServiceInstance> getInstanceResponse(List<ServiceInstance> instances) {
    if (instances.isEmpty()) return new EmptyResponse();
    for (ServiceInstance instance : instances) {
        if (NetUtil.localIpv4s().contains(instance.getHost())) {
            return new DefaultResponse(instance);
        }
    }
    return new DefaultResponse(instances.get(ThreadLocalRandom.current().nextInt(instances.size())));
}

Dynamic Gray Release Strategy

The gray strategy uses Nacos as the control center. Service versions are defined in application.yml and stored as metadata. Gray rules (target version and weight) are placed in Nacos configuration, e.g.:

gray:
  loadbalancer:
    enabled: true
    rules:
      ruoyi-system:
        version: v1.1.0
        weight: 20

A new GrayLoadBalancer reads both the instance list and the gray rule, splits instances into gray and normal groups, and routes traffic according to the configured weight:

private Response<ServiceInstance> getInstanceResponse(List<ServiceInstance> instances) {
    GrayRule grayRule = grayLoadBalancerProperties.getRules().get(serviceId);
    String grayVersion = grayRule.getVersion();
    int grayWeight = grayRule.getWeight();
    List<ServiceInstance> grayInstances = instances.stream()
        .filter(i -> grayVersion.equals(i.getMetadata().get("version")))
        .collect(Collectors.toList());
    List<ServiceInstance> normalInstances = instances.stream()
        .filter(i -> !grayVersion.equals(i.getMetadata().get("version")))
        .collect(Collectors.toList());
    int random = new Random().nextInt(100);
    if (random < grayWeight) {
        return new DefaultResponse(grayInstances.get(new Random().nextInt(grayInstances.size())));
    } else {
        return new DefaultResponse(normalInstances.get(new Random().nextInt(normalInstances.size())));
    }
}

Auto‑Configuration

The component is wired into Spring via GrayLoadBalancerAutoConfiguration, which is activated only when gray.loadbalancer.enabled=true:

@Configuration
@EnableConfigurationProperties(GrayLoadBalancerProperties.class)
@ConditionalOnProperty(value = "gray.loadbalancer.enabled", havingValue = "true")
public class GrayLoadBalancerAutoConfiguration {
    @Bean
    public ReactorLoadBalancer<ServiceInstance> grayLoadBalancer(Environment env,
            LoadBalancerClientFactory factory,
            GrayLoadBalancerProperties props) {
        String name = env.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
        return new GrayLoadBalancer(factory.getLazyProvider(name, ServiceInstanceListSupplier.class),
                name, props);
    }
}

Practical Verification

After deploying both the old (v1.0.0) and new (v1.1.0) versions of ruoyi-system, a simple Bash script sends 100 requests and counts the returned version strings:

#!/bin/bash
for i in {1..100}; do
  curl -s http://localhost:8080/system/user/profile/1 | jq .data.version
done | sort | uniq -c | sort -nr

Sample output shows roughly 80 % traffic to the old version and 20 % to the new version, confirming that the gray rule (weight 20) is effective. Adjusting the weight in Nacos gradually shifts traffic until it reaches 100 % for a full release.

Future Extensions

Routing based on request headers or parameters to target specific user groups.

Dynamic weight adjustment using real‑time performance metrics (CPU, latency, error rate).

The presented component demonstrates how to turn Spring Cloud LoadBalancer into an intelligent traffic‑shaping engine, enabling safe, zero‑downtime releases in microservice architectures.