How to Implement Full‑Chain Gray Release with OpenSergo and Apache APISIX

What Is Full‑Link Gray Release?

In a microservice architecture, services often depend on each other, and a new feature may require multiple services to be upgraded simultaneously. Full‑link gray release allows the new versions of several services to be validated with a small amount of traffic by isolating the entire request chain—from the gateway to backend services—so that only the gray‑marked traffic reaches the new versions.

How It Works

During deployment, only the gray versions of services are installed. As a request traverses the call chain, the gateway, middleware, and each microservice identify gray traffic and dynamically forward it to the corresponding gray version. The routing adapts in real time when service versions change, reducing machine costs and operational effort while enabling precise, real‑time traffic control.

Key Challenges

Dynamic routing based on traffic characteristics.

Grouping all nodes of a service and distinguishing their versions.

Tagging traffic and versions.

Identifying gray traffic of different versions.

OpenSergo Overview

OpenSergo is an open, language‑agnostic service‑governance standard that covers the entire distributed system—from gateway to database and cache—using a unified CRD/DSL configuration. It lets developers define governance rules once and apply them across Java, Go, Node.js, and other runtimes.

OpenSergo v1alpha1 Traffic‑Routing Standard

The standard consists of three parts:

WorkloadLabelRule – assigns labels to a group of workloads (e.g., APISIX upstreams).

TrafficLabelRule – tags incoming traffic based on request attributes.

Matching – routes traffic with specific workload and traffic labels to the appropriate workload.

Example: Tagging Traffic

The following YAML creates a TrafficLabelRule that marks requests with header X-User-Id equal to 12345 and path /index as gray:

apiVersion: traffic.opensergo.io/v1alpha1
kind: TrafficLabelRule
metadata:
  name: my-traffic-label-rule
  labels:
    app: my-app
spec:
  selector:
    app: my-app
  trafficLabel: gray
  match:
  - condition: "=="
    type: header
    key: 'X-User-Id'
    value: 12345
  - condition: "=="
    type: path
    value: "/index"

Example: Tagging Workloads

For Kubernetes, add labels to Pods in the Deployment template. For Nacos, set the environment variable traffic.opensergo.io/label: gray. A generic WorkloadLabelRule can also be used:

apiVersion: traffic.opensergo.io/v1alpha1
kind: WorkloadLabelRule
metadata:
  name: gray-sts-label-rule
spec:
  workloadLabels: ['gray']
  selector:
    app: my-app-gray

Traffic Coloring

Gray traffic is identified by adding a label at the request source (e.g., front‑end) or by the gateway. If a request reaches a gray node without a label, the system can automatically color it so subsequent hops know it belongs to the gray lane.

Implementing Full‑Link Gray Release with Apache APISIX

Below are the practical steps performed on Alibaba Cloud.

Prerequisite: Install Ingress‑APISIX

helm repo add apisix https://charts.apiseven.com
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
kubectl create ns ingress-apisix
helm install apisix apisix/apisix \
  --set gateway.type=LoadBalancer \
  --set ingress-controller.enabled=true \
  --set etcd.persistence.storageClass="alicloud-disk-ssd" \
  --set etcd.persistence.size="20Gi" \
  --namespace ingress-apisix \
  --set ingress-controller.config.apisix.serviceNamespace=ingress-apisix
kubectl get service --namespace ingress-apisix

After installation, the namespace contains the stateless APISIX and ingress‑controller pods and a stateful etcd pod.

Install APISIX Dashboard

helm repo add apisix https://charts.apiseven.com
helm repo update
helm install apisix-dashboard apisix/apisix-dashboard --namespace ingress-apisix

Access the dashboard at {slb‑ip}:9000 (default credentials admin/admin).

Enable MSE Service Governance

Activate MSE microservice governance and install the ack‑onepilot component. Detailed steps are provided in the Alibaba Cloud documentation.

Deploy Demo Applications

Deploy three services (A, B, C) each with a base and a gray version, plus a Nacos server for service discovery. After deployment, configure APISIX services to point to the appropriate upstreams.

Routing Rules Example

Route based on the env query parameter:

{
  "uri": "/*",
  "name": "spring-cloud-a",
  "methods": ["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],
  "host": "www.demo.com",
  "upstream_id": "401152455435354748",
  "labels": {"API_VERSION": "0.0.1"},
  "status": 1
}

Gray route (matches env=gray)

{
  "uri": "/*",
  "name": "spring-cloud-a-gray",
  "priority": 1,
  "methods": ["GET","POST","PUT","DELETE","PATCH","HEAD","OPTIONS","CONNECT","TRACE"],
  "host": "www.demo.com",
  "vars": [["arg_env","==","gray"]],
  "upstream_id": "401163331936715388",
  "labels": {"API_VERSION": "0.0.1"},
  "status": 1
}

Result Verification

Base environment request:

curl -H "Host:www.demo.com" http://47.97.253.177/a
# Output shows A → B → C (baseline IPs)

Gray environment request:

curl -H "Host:www.demo.com" http://47.97.253.177/a?env=gray
# Output shows Agray → Bgray → Cgray (gray IPs)

Note: 47.97.253.177 is the public IP of the APISIX gateway.

Summary

Full‑link gray release is a core capability for microservice modernization. By leveraging OpenSergo’s unified CRD standard and Apache APISIX’s flexible routing, developers can implement fine‑grained, multi‑service gray deployments without building separate environments, saving cost and operational effort. The approach also integrates with Alibaba Cloud MSE, supporting gateways, ALB, Dubbo, Spring Cloud, RocketMQ, and databases.