Mastering Canary Deployments with Ingress‑Nginx Annotations in Kubernetes

Background

When using Kubernetes as a cloud platform for business applications, blue‑green deployments are often needed for version iteration. Istio can be too heavyweight for simple traffic splitting, while Ingress‑Nginx 0.21 introduced a lightweight Canary feature that uses annotations to control traffic distribution among multiple backend services.

Ingress‑Nginx Canary Annotations

nginx.ingress.kubernetes.io/canary: "true"

– Enables Canary mode. nginx.ingress.kubernetes.io/canary-weight – Sets the percentage (0‑100) of traffic routed to the Canary service. nginx.ingress.kubernetes.io/canary-by-header – Routes traffic based on a request header value (useful for A/B testing). nginx.ingress.kubernetes.io/canary-by-header-value – Works with canary-by-header to match a specific header value. nginx.ingress.kubernetes.io/canary-by-cookie – Routes traffic based on a cookie value.

The priority order for rule evaluation is: canary-by-header → canary-by-cookie → canary-weight.

1. Weight‑Based Small‑Scale Test

Deploy version v1:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
  labels:
    app: echoserverv1
  name: echoserverv1
  namespace: echoserver
spec:
  rules:
  - host: echo.chulinx.com
    http:
      paths:
      - backend:
          serviceName: echoserverv1
          servicePort: 8080
        path: /
---
kind: Service
apiVersion: v1
metadata:
  name: echoserverv1
  namespace: echoserver
spec:
  selector:
    name: echoserverv1
  type: ClusterIP
  ports:
  - name: echoserverv1
    port: 8080
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: echoserverv1
  namespace: echoserver
  labels:
    name: echoserverv1
spec:
  template:
    metadata:
      labels:
        name: echoserverv1
    spec:
      containers:
      - image: mirrorgooglecontainers/echoserver:1.10
        name: echoserverv1
        ports:
        - containerPort: 8080
          name: echoserverv1

Verify resources with kubectl get pod,service,ingress -n echoserver and confirm all requests hit the v1 pod:

for i in `seq 10`; do curl -s echo.chulinx.com | grep Hostname; done

Deploy version v2 with a 50 % weight:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "50"
  labels:
    app: echoserverv2
  name: echoserverv2
  namespace: echoserver
spec:
  rules:
  - host: echo.chulinx.com
    http:
      paths:
      - backend:
          serviceName: echoserverv2
          servicePort: 8080
        path: /
---
kind: Service
apiVersion: v1
metadata:
  name: echoserverv2
  namespace: echoserver
spec:
  selector:
    name: echoserverv2
  type: ClusterIP
  ports:
  - name: echoserverv2
    port: 8080
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: echoserverv2
  namespace: echoserver
  labels:
    name: echoserverv2
spec:
  template:
    metadata:
      labels:
        name: echoserverv2
    spec:
      containers:
      - image: mirrorgooglecontainers/echoserver:1.10
        name: echoserverv2
        ports:
        - containerPort: 8080
          name: echoserverv2

After applying, kubectl get pod,service,ingress -n echoserver shows both versions. A ten‑request test typically yields ~5 requests to each version, demonstrating the weight‑based split.

2. Header‑Based A/B Testing

Update the v2 Ingress to add nginx.ingress.kubernetes.io/canary-by-header: "v2". Test three header values: v2:always – all traffic goes to v2. v2:never – all traffic stays with v1. v2:true – traffic follows the configured weight (≈50 %).

for i in `seq 10`; do curl -s -H "v2:always" echo.chulinx.com | grep Hostname; done
for i in `seq 10`; do curl -s -H "v2:never" echo.chulinx.com | grep Hostname; done
for i in `seq 10`; do curl -s -H "v2:true" echo.chulinx.com | grep Hostname; done

The results confirm the header‑driven routing behavior.

3. Custom Header‑Value Matching

Combine canary-by-header with canary-by-header-value: "true" to route only when the header value exactly matches “true”. Tests show that only the v2:true header triggers weight‑based routing, while other values fall back to the priority order.

for i in `seq 10`; do curl -s -H "v2:true" echo.chulinx.com | grep Hostname; done

4. Cookie‑Based Routing

Using nginx.ingress.kubernetes.io/canary-by-cookie, traffic can be split based on a cookie. Example commands:

for i in `seq 10`; do curl -s --cookie "user_from_shanghai" echo.chulinx.com | grep Hostname; done
for i in `seq 10`; do curl -s --cookie "user_from_shanghai:always" echo.chulinx.com | grep Hostname; done

These demonstrate that specific cookie values can force traffic to the Canary version.

Conclusion

Ingress‑Nginx Canary annotations provide a lightweight, flexible way to perform blue‑green deployments, canary releases, and A/B testing on Kubernetes without the overhead of a service mesh. By configuring weight, header, or cookie rules, operators can gradually shift traffic, validate new versions, and maintain system stability.