Mastering Ingress: Canary and Blue‑Green Deployments in Practice

Background

With the rise of micro‑service architectures, the number of services and release frequency increase, making full‑rollout risky. A failure in production can affect many users and require lengthy rollbacks. To minimize risk and maintain stability, teams adopt gray (canary) releases and blue‑green deployments.

What Is a Canary Release?

A canary (gray) release routes a small portion of traffic to a new version for testing. If the canary behaves correctly, the traffic share is gradually increased until the old version is fully replaced. The simplest method selects a random percentage of requests; more advanced approaches filter by request content, user attributes, or other criteria.

What Is a Blue‑Green Deployment?

A blue‑green deployment runs the old and new versions side‑by‑side, allowing zero‑downtime switches. Traffic is switched between the two versions by toggling routing weights (0 or 100). If an issue arises, the system can instantly roll back to the stable version.

Demo Application Deployment

First, deploy the initial version of the demo app:

$ cat demo.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo
  labels:
    app: demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo
  template:
    metadata:
      labels:
        app: demo
    spec:
      containers:
      - name: demo
        imagePullPolicy: Always
        image: registry.cn-shanghai.aliyuncs.com/kubesre01/demo:v1
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: demo-svc
spec:
  type: ClusterIP
  selector:
    app: demo
  ports:
  - port: 8080
    targetPort: 8080
$ kubectl apply -f demo.yml
deployment.apps/demo created

Deploy the new version:

$ cat demo_new.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-new
  labels:
    app: demo-new
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo-new
  template:
    metadata:
      labels:
        app: demo-new
    spec:
      containers:
      - name: demo-new
        imagePullPolicy: Always
        image: registry.cn-shanghai.aliyuncs.com/kubesre01/demo:v2
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: demo-new-svc
spec:
  type: ClusterIP
  selector:
    app: demo-new
  ports:
  - port: 8080
    targetPort: 8080
$ kubectl apply -f demo_new.yml
deployment.apps/demo_new created

Create the Ingress for the stable version:

$ cat demo-ingress.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo
spec:
  rules:
  - host: demo.kubesre.com
    http:
      paths:
      - path: /info
        pathType: Prefix
        backend:
          service:
            name: demo-svc
            port:
              number: 8080
  ingressClassName: nginx
$ kubectl apply -f demo-ingress.yml
ingress.networking.k8s.io/demo-ingress created
$ curl http://demo.kubesre.com/info
{"message":"云原生运维圈！"}

At this point, both the original and new versions are deployed.

Traffic Splitting by Client Request Header

To route requests that contain the header user=kubesre to the new version, create a canary Ingress with appropriate annotations:

$ cat demo-new-canary.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo-new-canary
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "user"
    nginx.ingress.kubernetes.io/canary-by-header-value: "kubesre"
spec:
  rules:
  - host: demo.kubesre.com
    http:
      paths:
      - path: /info
        pathType: Prefix
        backend:
          service:
            name: demo-new-svc
            port:
              number: 8080
  ingressClassName: nginx
$ kubectl apply -f demo-new-canary.yml
ingress.networking.k8s.io/demo-new-canary created

Verification:

# Header user:kubesre reaches the new version
curl -H "user: kubesre" http://demo.kubesre.com/info
{"message":"云原生运维圈！新版本"}
# Other requests reach the old version
curl http://demo.kubesre.com/info
{"message":"云原生运维圈！"}

Traffic Splitting by Client Source IP

To expose the new version only to internal IP 123.456.789.123, use the same canary Ingress but change the header annotation to inspect X-Forwarded-For:

$ cat demo-new-canary.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo-new-canary
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "X-Forwarded-For"
    nginx.ingress.kubernetes.io/canary-by-header-value: "123.456.789.123"
spec:
  rules:
  - host: demo.kubesre.com
    http:
      paths:
      - path: /info
        pathType: Prefix
        backend:
          service:
            name: demo-new-svc
            port:
              number: 8080
  ingressClassName: nginx
$ kubectl apply -f demo-new-canary.yml
ingress.networking.k8s.io/demo-new-canary created

Verification (simulated with a custom header):

# Simulated internal IP via header
curl -H "X-Forwarded-For:123.456.789.123" http://demo.kubesre.com/info
{"message":"云原生运维圈！新版本"}
# Other requests get the old version
curl http://demo.kubesre.com/info
{"message":"云原生运维圈！"}

Traffic Splitting by Service Weight

To send 20 % of traffic to the new version, set the canary-weight annotation:

$ cat demo-new-canary.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo-new-canary
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "20"
spec:
  rules:
  - host: demo.kubesre.com
    http:
      paths:
      - path: /info
        pathType: Prefix
        backend:
          service:
            name: demo-new-svc
            port:
              number: 8080
  ingressClassName: nginx
$ kubectl apply -f demo-new-canary.yml
ingress.networking.k8s.io/demo-new-canary created

Verification (run 20 requests and observe roughly 20 % responses from the new version):

for i in {1..20}; do curl http://demo.kubesre.com/info; done;

The output shows a mix of old and new version messages, matching the configured weight.

Annotation Details

nginx.ingress.kubernetes.io/canary-by-header – routes traffic based on a specific request header; useful for gray releases.

nginx.ingress.kubernetes.io/canary-by-header-value – works with the above to match a particular header value.

nginx.ingress.kubernetes.io/canary-by-header-pattern – similar to the value annotation but uses a regular expression; ignored if both are present.

nginx.ingress.kubernetes.io/canary-by-cookie – routes based on a cookie value (supports only “always” or “never”).

nginx.ingress.kubernetes.io/canary-weight – defines the percentage of traffic sent to the canary service; applicable to blue‑green deployments.

Annotations are evaluated in priority order: canary‑by‑header → canary‑by‑cookie → canary‑weight.

Conclusion

The article presented canary and blue‑green release strategies using Nginx Ingress, demonstrated concrete YAML manifests, and showed how to verify each traffic‑splitting method. Future sections will cover Ingress TLS certificate management and mutual authentication.