How to Implement Canary Releases with Nginx Ingress on Alibaba Cloud ACK

Gray (canary) release reduces production deployment risk and improves service stability, especially in fast‑iteration cloud‑native development. Compared with Kubernetes default rolling updates, traffic‑based canary verification is more precise.

Basic Principle

For a typical web service, Ingress can route a portion of traffic to a gray environment based on request headers or other identifiers. Requests matching the gray label are sent to the gray service, allowing verification before full rollout.

Architecture Overview

The traffic enters via the HOST defined in the Ingress. Nginx‑Ingress reads the canary annotation (e.g., nginx.ingress.kubernetes.io/canary: "true") and routes matching requests (identified by a header like _env: grey) to the gray service. Internal service‑to‑service calls do not carry the gray label, so the architecture only handles external traffic.

Practical Steps on Alibaba Cloud ACK

Import the ACK cluster and create an application in AppStack.

Define two environments sharing the same cluster: grey for canary and ack-prod for production.

Configure the deployment orchestration. The key is the canary annotation in the Ingress; for example, using header‑based routing with nginx.ingress.kubernetes.io/canary-by-header: "_env" and nginx.ingress.kubernetes.io/canary-by-header-value: "grey". The following template shows the conditional rendering:

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ .AppStack.appName }}-{{ .AppStack.envName }}
  namespace: {{ .Values.namespace }}
{{ if eq .AppStack.envName "grey" }}
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "_env"
    nginx.ingress.kubernetes.io/canary-by-header-value: "grey"
{{ end }}
spec:
  rules:
  - host: {{ .Values.host }}
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: {{ .AppStack.appName }}-{{ .AppStack.envName }}
            port:
              number: 80

Service and Deployment resources are also defined, referencing the same environment name.

---
apiVersion: v1
kind: Service
metadata:
  name: {{ .AppStack.appName }}-{{ .AppStack.envName }}
  namespace: {{ .Values.namespace }}
spec:
  selector:
    run: {{ .AppStack.appName }}-{{ .AppStack.envName }}
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .AppStack.appName }}-{{ .AppStack.envName }}
  namespace: {{ .Values.namespace }}
spec:
  replicas: {{ .Values.replicas }}
  selector:
    matchLabels:
      run: {{ .AppStack.appName }}-{{ .AppStack.envName }}
  template:
    metadata:
      labels:
        run: {{ .AppStack.appName }}-{{ .AppStack.envName }}
    spec:
      containers:
      - name: main
        image: {{ .AppStack.image.backend }}
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: {{ .Values.cpuLimit }}
            memory: {{ .Values.memoryLimit }}
          requests:
            cpu: {{ .Values.cpuRequest }}
            memory: {{ .Values.memoryRequest }}

CI/CD Pipeline (AppStack Flow)

The pipeline includes stages for building the Go image, uploading artifacts, pushing the Docker image to Alibaba Cloud registry, manual approval, gray deployment, gray validation, production deployment, and cleanup. Important snippets:

stages:
  build:
    jobs:
      go_build:
        steps:
          golang_build_step:
            with:
              goVersion: "1.20.x"
              run: |
                export GOPROXY=https://goproxy.cn
                make build
          acr_docker_build_step:
            with:
              dockerRegistry: "registry.cn-zhangjiakou.aliyuncs.com/docker007/demo-go-echo"
  approve:
    jobs:
      ops_validate:
        component: "ManualValidate"
  grey:
    jobs:
      grey_deploy_job:
        component: "AppStackFlowDeploy"
        with:
          environment: "grey"
      grey_validate:
        component: "ManualValidate"
  deploy:
    jobs:
      ack_deploy_job:
        component: "AppStackFlowDeploy"
        condition: succeed('grey.grey_validate')
        with:
          environment: "ack-prod"
  cleanup:
    jobs:
      cleanup_grey_env_job:
        component: "AppStackCleanEnv"
        condition: failed('grey.grey_validate') || succeed('deploy.ack_deploy_job')

Verification

After deployment, map the Ingress external IP to go.demo.prod in /etc/hosts and test with curl or httpie:

http -v http://go.demo.prod/version               # production
http -v http://go.demo.prod/version _env:grey    # gray

FAQ

Should production still use batch deployment? Yes, to keep risk low even after gray validation.

How to combine config and data changes? Include them as separate steps in the pipeline, applying data changes before deployment and config changes after.

How to ensure internal service calls respect the gray environment? Use separate namespaces for gray and production; services within the same namespace will call each other's gray versions.

Can other release types like Function Compute be integrated? Yes, add corresponding steps to the pipeline YAML.

For more details, refer to the official Alibaba Cloud documentation and the Ingress‑nginx canary examples.