Cut page load from 5 s to 500 ms: 12 Nginx tuning parameters

Reducing page latency from 5 seconds to 500 ms requires more than raising a few Nginx directives; total latency includes DNS, TCP, TLS, browser queue, static resources, Nginx queuing, upstream services, databases and network round‑trip. Nginx tuning is useful only when evidence shows Web‑layer constraints such as file‑descriptor exhaustion, mismatched connection models, inefficient static‑file delivery, unreasonable compression, or insufficient request buffering. It cannot accelerate slow SQL, downstream APIs or cross‑region networks.

Methodology

Using Ubuntu 22.04’s open‑source Nginx, the guide walks through twelve configuration parameters. The workflow is:

Establish a baseline.

Formulate a hypothesis.

Apply a single‑parameter change in a gray rollout.

Verify the effect.

Rollback if necessary.

Baseline measurement uses curl -w to capture time_connect, time_starttransfer, time_total and HTTP code, repeated in low‑ and high‑traffic windows and correlated with access logs, APM data, CPU/I/O and connection counts.

Parameter 1: worker_processes

Determines the number of worker processes. The open‑source build supports auto, which selects a count based on visible CPUs. In containers, CPU quotas, cpusets or NUMA, verify the actual result with ps -C nginx -o pid,ppid,psr,%cpu,cmd and taskset. Increase only when CPU is saturated and upstream latency is normal; monitor request latency, CPU steal, context switches and per‑worker load balance.

Parameter 2: worker_cpu_affinity

Binds workers to specific CPUs via a bitmask. Modern Linux schedulers usually handle ordinary web loads; an incorrect mask or a changing container CPU set can cause contention. Use only in fixed‑core, clearly‑observed CPU environments; verify with taskset -pc and ensure each worker’s CPU usage is balanced before keeping the setting.

Parameter 3: worker_rlimit_nofile

Raises the hard limit for file descriptors that a worker may request, but cannot exceed the systemd or shell hard limit. Adjust only when the error log shows “too many open files” and after confirming the three‑layer limits (system, service, worker) are below the expected concurrent connections.

Parameter 4: worker_connections

Sets the maximum simultaneous connections per worker. Because each proxied request occupies at least a client and an upstream socket, the product worker_processes × worker_connections is not the total serviceable request count. Derive the value from peak connection counts, available file descriptors and memory; verify with ss -Htan state established and search error logs for “worker_connections are not enough” or “too many open files”.

Parameter 5: multi_accept

When on, a worker accepts as many new connections as possible in one go, which can help short‑burst spikes but may cause a single worker to monopolize connections. Keep it off by default and enable only after observing accept‑queue backlog and confirming improvement in a gray rollout.

Parameter 6: use epoll

Linux Nginx automatically selects epoll. Explicitly setting it is mainly for configuration clarity or to rule out interference; remove it if startup or reload fails. It does not affect upstream response latency.

Parameter 7: keepalive_timeout

Controls how long idle keep‑alive connections are kept. A longer timeout reduces TCP/TLS handshakes but can let slow clients hold workers and file descriptors; a shorter timeout increases handshakes and TIME_WAIT. Adjust in coordination with the load balancer’s idle timeout, CDN settings and HTTP/2 connection model, and monitor established connections, TIME_WAIT, 4xx/5xx rates and TLS handshake speed.

Parameter 8: keepalive_requests

Limits the maximum number of requests a single keep‑alive connection may handle. Raising it reduces connection churn, but an excessively high value can let long‑lived connections monopolize resources. Verify with access‑log analysis of 5xx trends and ensure changes do not cause spikes in TLS handshakes or client errors.

Parameter 9: sendfile & tcp_nopush

Enables kernel‑mode static file delivery and optimises packet framing. Suitable for local reliable filesystems; validate with

curl -sS -D /tmp/headers.txt -r 0-1023 https://example.com/static/file

and check Content‑Length, Content‑Range and download integrity. Do not apply to NFS, FUSE or paths that are frequently replaced.

Parameter 10: tcp_nodelay

Disables Nagle’s algorithm, reducing latency for small responses and API calls. Typical web services can start with on, but must be measured for TTFB, packet volume and CPU usage. It does not fix network loss or upstream bottlenecks.

Parameter 11: client_header_buffer_size & large_client_header_buffers

Defines how much header data Nginx can read. Too small yields 400/414 errors; too large inflates memory usage under high concurrency. Use the example client_header_buffer_size 1k; and large_client_header_buffers 4 8k; as a starting point, then adjust based on actual Cookie, JWT or proxy‑added header lengths. Monitor error logs for “client sent too long header line”.

Parameter 12: gzip

Compresses textual responses to reduce bandwidth at the cost of CPU. Enable with gzip on;, gzip_comp_level 4;, gzip_min_length 1024;, gzip_vary on;, gzip_proxied any; and a whitelist of types (e.g.,

text/plain text/css application/json application/javascript application/xml image/svg+xml

). Verify that upstream or CDN does not already compress, and watch CPU, TTFB and download size.

Backup, Gray Deployment and Rollback

Before any change, back up the current configuration, e.g.

cp -a /etc/nginx /var/backups/nginx-tune-20260730/etc-nginx-before

, and dump the effective config with

nginx -T > /var/backups/nginx-tune-20260730/nginx-T-before.txt

. Ensure the configuration is managed by Ansible, Helm, GitOps, etc., and that at least one instance can be rolled out gradually. Apply the new include file (e.g., /etc/nginx/conf.d/20-performance.conf) and run nginx -t before systemctl reload nginx. After reload, perform health‑check curls, verify key pages, APIs, TLS handshakes and error‑log health. If nginx -t fails or health checks break, stop, restore the backup and reload again.

Evidence of Effectiveness

Effective tuning must be demonstrated by comparing two sets of metrics for the same URL set: TTFB, total time, status codes and response size; Nginx 4xx/5xx, request_time, upstream_response_time; worker CPU/RSS, file‑descriptor usage, established/TIME_WAIT connections, retransmissions and disk I/O. Example Prometheus queries:

# Nginx process file‑descriptor usage
process_open_fds{job="nginx"}
/process_max_fds{job="nginx"}
# Node TCP established connections
node_netstat_Tcp_CurrEstab

Conclusions must be factual, e.g., “after the gray window no FD‑exhaustion errors appeared and tail latency remained stable”.

Interaction Risks

Parameters interact: increasing worker_connections without raising the FD limit merely moves the bottleneck; extending keepalive_timeout can reduce handshakes but increase FD and memory consumption; enabling gzip saves bandwidth but may raise CPU and TTFB if the CPU is already saturated. Change only one related group at a time to isolate cause‑effect relationships.

Final Note

Distinguish Nginx request time from the user‑perceived page load. If HTML TTFB is stable but the page remains slow, investigate resource waterfalls, caching headers, HTTP/2 multiplexing, CDN hit rates, front‑end bundle size and third‑party scripts. Further optimisation should then shift to application code, caching layers or network topology rather than additional Nginx knobs.