Mastering Canary Deployments with ingress-nginx: A Step-by-Step Guide

In daily work, applications are frequently upgraded; common release strategies include rolling update, blue‑green, and canary (gray) release.

Rolling update: replace old instances one by one until all are updated.

Blue‑green release: maintain two independent environments (green for production, blue for the new version) and switch traffic after testing.

Canary (gray) release: run both stable and canary versions in the same cluster, route a portion of traffic to the canary version, and promote it after verification.

Here we share how to achieve canary release using the ingress-nginx controller.

How to implement canary release with ingress-nginx

ingress-nginx is the official Kubernetes ingress controller, built on Nginx and extended with Lua plugins.

Canary release is realized by defining annotations on the Ingress resource. Supported rules are: nginx.ingress.kubernetes.io/canary-by-header: traffic split based on a request header. When the header value is always, all requests go to the canary; when never, none go to the canary. nginx.ingress.kubernetes.io/canary-by-header-value: the specific header value that triggers routing to the canary. Must be used together with the previous annotation. nginx.ingress.kubernetes.io/canary-weight: traffic split based on weight (0‑100%). Weight 0 disables the canary, weight 100 sends all traffic to it. nginx.ingress.kubernetes.io/canary-by-cookie: traffic split based on a cookie value, similar to the header rule.

Our implementation follows these steps:

Deploy two sets of resources in the cluster: a stable version and a canary version, each with its own Service.

Create two Ingress objects—one normal and one with the canary annotations.

After the canary version is verified, promote it to stable and remove the old stable resources.

Release scenario examples

Weight‑based release

Suppose version A is running in production and a bug‑fix version A2 is ready. Instead of switching all traffic at once, route 10% of traffic to A2, monitor it, then gradually increase the share.

Annotations needed for this scenario:

nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "10"

The canary annotation enables the rule, and canary-weight sets the traffic proportion (10%).

User‑request‑based release

Weight‑based routing cannot target specific users. For example, only users from Sichuan should see version A2 while other regions continue using version A.

Annotations for header‑based routing:

nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-by-header: "region"
nginx.ingress.kubernetes.io/canary-by-header-value: "sichuan"

When the request includes the header region: sichuan, it is routed to the canary service.

Concrete implementation

Two Docker images are prepared:

registry.cn-hangzhou.aliyuncs.com/rookieops/go-test:v1 (stable)

registry.cn-hangzhou.aliyuncs.com/rookieops/go-test:v2 (canary)

Deploy the stable version:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-server-stable
spec:
  selector:
    matchLabels:
      app: go-test
      version: stable
  replicas: 1
  template:
    metadata:
      labels:
        app: go-test
        version: stable
    spec:
      containers:
      - name: app-server
        image: registry.cn-hangzhou.aliyuncs.com/rookieops/go-test:v1
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: app-server-stable-svc
spec:
  selector:
    app: go-test
    version: stable
  ports:
  - name: http
    port: 8080

Deploy the canary version:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-server-canary
spec:
  selector:
    matchLabels:
      app: go-test
      version: canary
  replicas: 1
  template:
    metadata:
      labels:
        app: go-test
        version: canary
    spec:
      containers:
      - name: app-server
        image: registry.cn-hangzhou.aliyuncs.com/rookieops/go-test:v2
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: app-server-canary-svc
spec:
  selector:
    app: go-test
    version: canary
  ports:
  - name: http
    port: 8080

Ingress for the stable service:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: app-server-stable-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: joker.coolops.cn
    http:
      paths:
      - backend:
          serviceName: app-server-stable-svc
          servicePort: 8080

Ingress for the canary service (weight‑based example):

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: app-server-canary-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"
spec:
  rules:
  - host: joker.coolops.cn
    http:
      paths:
      - backend:
          serviceName: app-server-canary-svc
          servicePort: 8080

Testing with curl shows the expected traffic split (approximately 9:1 for the weight‑based case) and header‑based routing when the region: sichuan header is supplied.

Canary release pipeline design

Typical pipeline steps:

Deploy the canary version for testing.

After verification, replace the stable version with the canary image.

Delete the canary Ingress configuration.

Remove the old stable deployment.

Naming conventions are recommended for clarity:

Pipeline names: <APP_NAME>-stable and <APP_NAME>-canary Deployment names: <APP_NAME>-stable and <APP_NAME>-canary Service names: <APP_NAME>-stable-svc and <APP_NAME>-canary-svc Ingress names: <APP_NAME>-stable-ingress and <APP_NAME>-canary-ingress Two Jenkinsfiles (canary.Jenkinsfile and stable.Jenkinsfile) are used to deploy the respective versions. After the canary pipeline finishes, the cluster shows the canary pod, service, and ingress. Once the stable pipeline runs, only the stable resources remain.

Recommendation

For a more automated and user‑friendly experience, consider using argo‑rollouts together with ArgoCD . argo‑rollouts provides custom CRDs to control the rollout process, while ArgoCD offers GitOps‑based continuous delivery with a UI, reducing manual steps and improving safety.