Why Ingress‑NGINX Crashed: Hidden TLS Secret Pitfall and How to Fix It

1. Problem Background

One morning the testing environment became unreachable; the ingress-nginx-controller pod entered CrashLoopBackOff and kept restarting. Logs showed errors related to the xxapi.test.xxx.com ingress configuration. Deleting the problematic ingress resource allowed the controller to restart and the services to recover.

2. Fault Analysis

Recreating the ingress from the exported YAML reproduced the issue. The failure conditions were:

Ingress started with an HTTPS secret that contained an invalid certificate (e.g., the default service-account-token secret).

Some hosts were defined in spec.rules[*].host but not listed in spec.tls[*].hosts.

The cluster used ingress-nginx-controller image 0.32.0. The following sections analyze the controller startup flow.

2.1 Ingress NGINX Entry Function

// path: cmd/nginx/main.go
func main() {
    // parse flags, create client, instantiate controller, start it
    go ngx.Start()
    handleSigterm(ngx, func(code int) { os.Exit(code) })
}

2.2 NGINX Controller Instantiation

The controller initializes the Kubernetes client, creates the store (local cache), sets up the sync queue, and watches configuration files.

// path: internal/ingress/controller/nginx.go
func NewNGINXController(config *Configuration, mc metric.Collector) *NGINXController {
    // instantiate controller, store, syncQueue, etc.
    n.store = store.New(...)
    n.syncQueue = task.NewTaskQueue(n.syncIngress)
    // watch template changes
    _, err = watch.NewFileWatcher(nginx.TemplatePath, onTemplateChange)
    return n
}

2.3 NGINX Controller Startup

Main steps include starting the store informers, leader election, launching the NGINX process, and running the sync queue.

// path: internal/ingress/controller/nginx.go
func (n *NGINXController) Start() {
    n.store.Run(n.stopCh)
    setupLeaderElection(...)
    n.start(cmd)
    go n.syncQueue.Run(time.Second, n.stopCh)
    for {
        select {
        case err := <-n.ngxErrCh:
            // respawn NGINX if needed
        case event := <-n.updateCh.Out():
            // enqueue events to syncQueue
        case <-n.stopCh:
            return
        }
    }
}

2.4 syncIngress Analysis

The crash originates from extractTLSSecretName inside the syncIngress call chain.

// path: internal/ingress/controller/controller.go
func (n *NGINXController) syncIngress(obj interface{}) error {
    ings := n.store.ListIngresses(nil)
    hosts, servers, pcfg := n.getConfiguration(ings)
    // ...
}

The function iterates over ing.Spec.TLS. If a secret contains a non‑TLS certificate, cert.Certificate is nil, causing a panic at cert.Certificate.VerifyHostname(host), which crashes the controller.

3. Solution

Two mitigation directions:

At the platform level, filter out non‑TLS secrets when users select certificates for an ingress.

In code, add a nil check before calling VerifyHostname:

// path: internal/ingress/controller/controller.go
func extractTLSSecretName(host string, ing *ingress.Ingress, getLocalSSLCert func(string) (*ingress.SSLCert, error)) string {
    // ...
    for _, tls := range ing.Spec.TLS {
        secrKey := fmt.Sprintf("%v/%v", ing.Namespace, tls.SecretName)
        cert, err := getLocalSSLCert(secrKey)
        if cert == nil || cert.Certificate == nil {
            continue
        }
        err = cert.Certificate.VerifyHostname(host)
        return tls.SecretName
    }
    return ""
}

We have also opened an issue on the upstream ingress-nginx repository to request a permanent fix.

By applying these changes, the controller no longer crashes when a non‑TLS secret is mistakenly referenced.