Supercharge Nginx Ingress Performance with Kernel and Config Tweaks

Overview

Nginx Ingress Controller implements the Kubernetes Ingress API using Nginx, a high‑performance gateway. To fully exploit its performance, both kernel and Nginx configuration parameters must be tuned.

Kernel Parameter Tuning

Adjust the following kernel settings to maximize Ingress performance under high load.

Increase the TCP listen backlog

The effective backlog size is the minimum of

somaxconn

and the Nginx

backlog

directive. Nginx defaults to 511, while the Ingress controller reads

somaxconn

(default 4096). Set it to a higher value, e.g., 65535.

<code>sysctl -w net.core.somaxconn=65535</code>

Expand the source port range

High concurrency can exhaust source ports. Expand

net.ipv4.ip_local_port_range

to cover the full range.

<code>sysctl -w net.ipv4.ip_local_port_range="1024 65535"</code>

Enable TIME_WAIT reuse

Allow sockets in TIME_WAIT to be reused, reducing port exhaustion.

<code>sysctl -w net.ipv4.tcp_tw_reuse=1</code>

Also shorten FIN_WAIT2 and TIME_WAIT timers:

<code>sysctl -w net.ipv4.tcp_fin_timeout=15
sysctl -w net.netfilter.nf_conntrack_tcp_timeout_time_wait=30</code>

Increase max TIME_WAIT connections

Raise

net.ipv4.tcp_max_tw_buckets

to handle more simultaneous TIME_WAIT sockets.

<code>sysctl -w net.ipv4.tcp_max_tw_buckets=55000</code>

Raise the maximum number of file handles

Nginx needs two file descriptors per request (client and upstream). Increase

fs.file-max

to allow more concurrent connections.

<code>sysctl -w fs.file-max=1048576</code>

Kernel tuning example

<code>initContainers:
  - name: setsysctl
    image: busybox
    securityContext:
      privileged: true
    command:
    - sh
    - -c
    - |
      sysctl -w net.core.somaxconn=65535
      sysctl -w net.ipv4.ip_local_port_range="1024 65535"
      sysctl -w net.ipv4.tcp_max_tw_buckets=55000
      sysctl -w net.ipv4.tcp_tw_reuse=1
      sysctl -w fs.file-max=1048576
      sysctl -w net.ipv4.tcp_fin_timeout=15
      sysctl -w net.netfilter.nf_conntrack_tcp_timeout_time_wait=30</code>

Application‑Layer Configuration Tuning

Increase keep‑alive request limit

Set

keepalive_requests

to a high value (e.g., 10000) so a single keep‑alive connection can serve many requests, reducing the number of TIME_WAIT sockets.

Increase keep‑alive idle connections for upstream

Adjust

upstream‑keepalive‑connections

(default 32) to a larger number such as 1000 to accommodate bursty traffic.

Gateway timeout settings

Fine‑tune the following timeouts to quickly drop stalled requests:

proxy-connect-timeout

: set to 3 seconds

proxy-read-timeout

: set to 3 seconds

proxy-send-timeout

: set to 3 seconds

Raise max worker connections

Increase

max-worker-connections

from the default 16384 to, for example, 65536.

Optimize retry mechanism

Disable upstream retry in the Ingress controller (

proxy-next-upstream="off"

) and limit retries in the edge Nginx using

proxy_next_upstream_tries

to avoid cascading retries.

Enable Brotli compression

Activate Brotli for better compression ratios than gzip. Set the following ConfigMap keys:

enable-brotli: "true"

brotli-level: "6"

(range 1‑11)

brotli-types: "..."

(list of MIME types)

ConfigMap example

<code>apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-ingress-controller

data:
  keep-alive-requests: "10000"
  upstream-keepalive-connections: "200"
  max-worker-connections: "65536"
  proxy-connect-timeout: "3"
  proxy-read-timeout: "3"
  proxy-send-timeout: "3"
  proxy-next-upstream: "off"
  enable-brotli: "true"
  brotli-level: "6"
  brotli-types: "text/xml image/svg+xml application/x-font-ttf image/vnd.microsoft.icon application/x-font-opentype application/json font/eot application/vnd.ms-fontobject application/javascript font/otf application/xml application/xhtml+xml text/javascript application/x-javascript text/plain application/x-font-truetype application/xml+rss image/x-icon font/opentype text/css image/x-win-bitmap"</code>

References

Optimizing nginx‑ingress‑controller concurrency: https://cloud.tencent.com/developer/article/1537695

Nginx Ingress ConfigMap reference: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/

Tuning NGINX for Performance: https://www.nginx.com/blog/tuning-nginx/

ngx_http_upstream_module documentation: http://nginx.org/en/docs/http/ngx_http_upstream_module.html