Choosing an Ingress Controller: Production Comparison of NGINX, Traefik, and APISIX

Choosing an Ingress controller should start from the team’s traffic, change, and failure models because the same Ingress object can behave differently under different controllers.

Comparable capability layers

Northbound access : need HTTP, HTTPS, gRPC, TCP, UDP or Gateway API? Common mistake – mixing layer‑4 exposure with HTTP routing.

Data plane : who handles high concurrency, long connections, rewrites, authentication, rate‑limit, canary releases? Common mistake – measuring only QPS without checking rule paths.

Control plane : how are routes published, rolled back, audited and isolated? Common mistake – assuming a successful Helm install means the controller is operable.

Operations : how are logs, metrics, certificates, upgrades and emergency traffic cuts handled? Common mistake – adding monitoring only after a failure.

NGINX Ingress – standard HTTP proxy governance

Best for teams already familiar with NGINX that need standard HTTP/HTTPS, gRPC, rewrites, basic auth, TLS termination and limited canary releases. The controller combines Ingress objects with a ConfigMap to generate NGINX configuration and reloads on changes. Annotation‑driven policies can become hard to audit.

# Conservative ConfigMap baseline for ingress-nginx
apiVersion: v1
kind: ConfigMap
metadata:
  name: <controller-release-name>-controller
  namespace: <controller-namespace>

data:
  allow-snippet-annotations: "false"
  use-forwarded-headers: "true"
  compute-full-forwarded-for: "true"
  proxy-body-size: "10m"
  proxy-read-timeout: "60"
  proxy-send-timeout: "60"

Disabling snippet annotations reduces the risk of arbitrary NGINX code. Forwarded‑header handling must be trusted; body size and timeouts are service‑level contracts, not performance knobs.

Traefik – routing and middleware as first‑class objects

Ideal for teams that prefer declarative, cloud‑native configuration and want dynamic service discovery with reusable middleware. Traefik uses IngressRoute and Middleware CRDs, keeping routing logic separate from annotation clutter.

# Minimal values.yaml for Traefik
providers:
  kubernetesIngress:
    enabled: true
    ingressClass: traefik
  kubernetesCRD:
    enabled: true
    ingressClass:
      enabled: true
      isDefaultClass: false
metrics:
  prometheus:
    enabled: true
service:
  type: LoadBalancer

# Example Middleware for rate limiting
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: api-rate-limit
  namespace: <namespace>
spec:
  rateLimit:
    average: 50
    burst: 100
    period: 1s

# IngressRoute that references the middleware
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: echo-traefik
  namespace: <namespace>
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`<domain>`) && PathPrefix(`/`)
      kind: Rule
      middlewares:
        - name: api-rate-limit
      services:
        - name: echo-backend
          port: 80
  tls:
    secretName: <tls-secret>

APISIX – when the ingress also acts as an API gateway

Fits scenarios that require consumer management, JWT/OIDC, fine‑grained rate limiting, request rewriting, canary releases and audit. APISIX introduces its own CRDs ( ApisixRoute, ApisixPluginConfig) and stores configuration in etcd, so the control‑plane HA and plugin whitelist become critical.

# Sample ApisixRoute HTTP rule
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: echo-apisix
  namespace: <namespace>
spec:
  http:
  - name: echo
    match:
      hosts:
      - <domain>
      paths:
      - "/*"
    backends:
    - serviceName: echo-backend
      servicePort: 80
    plugin_config_name: api-limit

# Plugin config for limit‑count
apiVersion: apisix.apache.org/v2
kind: ApisixPluginConfig
metadata:
  name: api-limit
  namespace: <namespace>
spec:
  plugins:
  - name: limit-count
    enable: true
    config:
      count: 100
      time_window: 60
      key: remote_addr
      rejected_code: 429

Pre‑deployment fact‑finding

Collect version, node and Ingress object information before replacing or adding a controller.

# Record client, cluster and node basics
kubectl -n <namespace> version --client --output=yaml
kubectl -n <namespace> config current-context
kubectl get nodes -n <namespace> -o wide
helm version --short

# Export current Ingresses for comparison
kubectl get ingress -n <namespace> -o yaml > <backup-dir>/ingress.before.yaml
kubectl get ingressclass -n <namespace> -o yaml
kubectl get deployment -n <controller-namespace> -o wide

Testing and validation

Deploy a minimal echo backend and verify routing, TLS, health checks and metrics.

# Verify HTTPS health endpoint
curl --fail --silent --resolve "${domain}:443:${entry_ip}" "https://${domain}/healthz" -o /dev/null
# Check key response headers
curl --head --resolve "${domain}:443:${entry_ip}" "https://${domain}/" | grep -iE '^(HTTP/|server:|strict-transport-security:|content-type:)'

Prometheus queries differ per controller.

# NGINX 5xx error ratio
sum(rate(nginx_ingress_controller_requests{status=~"5.."}[5m])) /
sum(rate(nginx_ingress_controller_requests[5m]))

# Traefik P95 request duration
histogram_quantile(0.95, sum by (le) (rate(traefik_service_request_duration_seconds_bucket[5m])))

Rollout, gray‑release and rollback

Create a new IngressClass and controller service, migrate low‑risk services first, verify logs, metrics and alerts, then migrate the rest. Rollback should apply a previously backed‑up manifest rather than hand‑crafting reverse changes.

# Rollback a single Ingress from backup
kubectl diff -n <namespace> -f <backup-dir>/echo-nginx.before.yaml || true
kubectl apply -n <namespace> -f <backup-dir>/echo-nginx.before.yaml

Decision matrix

Condition: Existing mature NGINX ops, standard HTTP routing → More suitable: NGINX Ingress (open questions: annotation governance, snippet policy, upgrade compatibility)

Condition: Need CRD routing and composable middleware, cloud‑native declarative style → More suitable: Traefik (open questions: IngressRoute vs Ingress boundaries, certificate workflow, metric definitions)

Condition: Ingress must act as full API gateway (auth, quota, plugins) → More suitable: APISIX (open questions: control‑plane HA, plugin whitelist, permission isolation, config store recovery)

Production readiness is defined by unique route ownership, auditable changes, observable failures, explainable metrics, executable gray‑release and rehearsed rollback – not merely by the controller being installed.