How to Supercharge Nginx Performance: Network, Hardware, OS, and Config Tuning

Factors Influencing Nginx Performance

Nginx performance depends on network, hardware, operating‑system settings and Nginx configuration itself.

Network Layer

Bandwidth : Higher upstream bandwidth raises the maximum concurrent connections.

Network quality : Packet loss, retransmissions and high latency increase response time.

Switch convergence ratio : A ratio > 1 means downstream traffic is blocked in the switch, reducing proxy throughput.

Network architecture : Prefer Nginx or a layer‑4 load balancer as the first hop; avoid firewalls as the entry point. Use multiple ISP links with policy routing when possible and design a “fat‑tree” topology for large deployments.

Server Hardware Layer

CPU : Nginx uses a master‑worker model; the number of CPU cores determines the optimal number of worker processes.

Memory : More RAM raises the theoretical maximum connections and enables caching via tmpfs.

Disk : Place logs and temporary files on SSD/PCIe‑SSD to minimise I/O latency.

NIC : Use high‑performance NICs (e.g., Intel i350 for 1 GbE, Mellanox for 10 GbE) and bind interrupts to separate CPU cores.

Operating‑System Layer

Key /etc/sysctl.conf parameters for high‑throughput Nginx deployments:

net.ipv4.tcp_max_tw_buckets = 1000
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_rmem = 4096 8388608 16777216
net.ipv4.tcp_wmem = 4096 8388608 16777216
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.somaxconn = 262144
net.ipv4.tcp_max_orphans = 32768
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_synack_retries = 1
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_retries2 = 5
net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_tw_reuse = 0
net.ipv4.ip_local_port_range = 10000 65535
net.ipv4.tcp_slow_start_after_idle = 0

Raise file‑handle limits ( fs.file-max, fs.nr_open) to match the expected connection count.

Disable irqbalance for Nginx‑heavy workloads to avoid interrupt skew.

Stop unnecessary services (e.g., sendmail, bind) to prevent resource exhaustion.

Nginx Layer

Compilation Optimization

Compile only the modules required for the target workload. Example for a static‑resource server:

./configure \
  --prefix=/opt/websuite/nginx \
  --conf-path=/opt/config/nginx/nginx.conf \
  --modules-path=/opt/websuite/nginx/modules \
  --error-log-path=/opt/logs/nginx/error.log \
  --http-log-path=/opt/logs/nginx/access.log \
  --pid-path=/opt/run/nginx --user=websuite \
  --group=websuite \
  --with-file-aio \
  --with-http_ssl_module \
  --with-http_v2_module \
  --with-http_stub_status_module \
  --without-http_ssi_module \
  --without-http_charset_module \
  --without-http_access_module \
  --without-http_auth_basic_module \
  --without-http_autoindex_module \
  --without-http_geo_module \
  --without-http_split_clients_module \
  --without-http_proxy_module \
  --without-http_fastcgi_module \
  --without-http_uwsgi_module \
  --without-http_scgi_module \
  --without-http_memcached_module \
  --without-http_empty_gif_module \
  --without-http_browser_module \
  --without-http_upstream_hash_module \
  --without-http_upstream_ip_hash_module \
  --without-http_upstream_least_conn_module \
  --without-http_upstream_keepalive_module \
  --without-http_upstream_zone_module \
  --http-client-body-temp-path=/opt/websuite/nginx/temp/client \
  --without-mail_pop3_module \
  --without-mail_imap_module \
  --without-mail_smtp_module \
  --with-google_perftools_module \
  --with-pcre=/tmp/nginx/pcre-8.41 \
  --with-pcre-jit \
  --with-openssl=/tmp/nginx/openssl-1.0.2j \
  --with-openssl-opt="threads shared no-zlib no-comp no-ssl2 no-ssl3 no-ssl3-method"

Log Optimization

Increase the log buffer, filter unnecessary URIs with map, and define an efficient log_format:

map $uri $expanded_name {
    ~^(.*\.(gif|jpg|jpeg|png|bmp|swf|js|css)$) 0;
    default 1;
}
access_log /opt/logs/openresty/access.www.grapenvine.cn \
    proxy buffer=1m if=$expanded_name;

Proxy & FastCGI Tuning

proxy_connect_timeout   30;
proxy_send_timeout      30;
proxy_read_timeout      60;
proxy_buffer_size       64k;
proxy_buffers           4 64k;
proxy_busy_buffers_size 128k;
proxy_next_upstream     invalid_header http_500 http_503 http_403 http_502 http_504;
proxy_next_upstream_timeout 1s;
proxy_next_upstream_tries   1;

Set proxy_buffer_size based on the average $bytes_sent; set proxy_buffers according to the maximum value.

Main Configuration

worker_processes auto;
worker_cpu_affinity auto;
worker_rlimit_nofile 100000;
 pcre_jit on;
 events {
    use epoll;
    worker_connections 8192;
    accept_mutex off;
 }

Ensure

worker_processes * worker_connections < worker_rlimit_nofile

. Turn accept_mutex off for high traffic.

Default Server and reuseport

Configure a default server to set backlog and enable reuseport for better load distribution:

server {
    listen 80 default_server reuseport backlog=511;
    # other directives …
}

Load Balancing & Keepalive

Enable keepalive in upstream blocks for both proxy and FastCGI:

upstream netemu {
    server unix:/opt/run/php/pool1.sock;
    server unix:/opt/run/php/pool4.sock;
    keepalive 4;
}

For proxy:

proxy_http_version 1.1;
proxy_set_header Connection "";

For FastCGI:

fastcgi_keep_conn on;

Optional Third‑Party Modules

Nginx‑VTS : JSON status output for monitoring tools.

ngx_dynamic_upstream / nginx‑upsync‑module / lua‑upstream‑nginx‑module : Dynamic upstream updates for zero‑downtime deployments.

testcookie‑nginx‑module : Mitigates abusive bot traffic without Lua.

srcache‑nginx‑module + memcached / Redis / Couchbase : Distributed shared cache.

Frequently Asked Questions

Can Nginx cache dynamic content? Yes, with conditional rules (e.g., cache GET, bypass POST).

Does the new version support TCP? It does, but using Nginx for layer‑4 load balancing is not recommended.

Can Nginx cache JSP‑generated data? Yes, with appropriate Cache‑Control settings.

Which module handles WebSocket traffic? The nchan module is suitable.