Master Nginx Performance: Worker, CPU Affinity, Caching & System Tweaks

Worker Processes and CPU Affinity

Set worker_processes to the number of CPU cores or cores × 2. Use worker_cpu_affinity to bind each worker to a specific CPU core, e.g., for 4 cores: worker_cpu_affinity 0001 0010 0100 1000, and for 8 cores:

worker_cpu_affinity 00000001 00000010 00000100 00001000 00010000 00100000 01000000 10000000

. More than 8 workers rarely improve performance and may reduce stability.

File Descriptor Limits

Increase the maximum open files per Nginx process with worker_rlimit_nofile 65535; and ensure the system limit ( ulimit -n) matches. Configure limits in /etc/security/limits.conf:

* soft nofile 65535
* hard nofile 65535

Event Model

Use the high‑performance epoll model:

events {
    use epoll;
    worker_connections 65535;
    multi_accept on;
}

worker_connections

defines the maximum connections per worker; total connections equal worker_processes × worker_connections. multi_accept can be turned off for high‑throughput servers.

Efficient Transfer Settings

Enable fast file transmission:

http {
    include mime.types;
    default_type application/octet-stream;
    sendfile on;
    tcp_nopush on;
}

sendfile

uses the kernel sendfile() call; tcp_nopush reduces packet fragmentation when sendfile is on.

Connection Timeouts

keepalive_timeout 60;
tcp_nodelay on;
client_header_buffer_size 4k;
open_file_cache max=102400 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 1;
client_header_timeout 15;
client_body_timeout 15;
reset_timedout_connection on;
send_timeout 15;
server_tokens off;
client_max_body_size 10m;

These settings protect server resources, control idle connections, and limit request sizes.

FastCGI Tuning

fastcgi_connect_timeout 600;
fastcgi_send_timeout 600;
fastcgi_read_timeout 600;
fastcgi_buffer_size 64k;
fastcgi_buffers 4 64k;
fastcgi_busy_buffers_size 128k;
fastcgi_temp_file_write_size 128k;
fastcgi_temp_path /usr/local/nginx1.10/nginx_tmp;
fastcgi_intercept_errors on;
fastcgi_cache_path /usr/local/nginx1.10/fastcgi_cache levels=1:2 keys_zone=cache_fastcgi:128m inactive=1d max_size=10g;
fastcgi_cache cache_fastcgi;
fastcgi_cache_valid 200 302 1h;
fastcgi_cache_valid 301 1d;
fastcgi_cache_valid any 1m;
fastcgi_cache_min_uses 1;
fastcgi_cache_key http://$host$request_uri;
fastcgi_pass ...;

FastCGI caching reduces load on PHP and MySQL by storing generated content.

Gzip Compression

gzip on;
gzip_min_length 1k;
gzip_buffers 4 32k;
gzip_http_version 1.1;
gzip_comp_level 6;
gzip_types text/css text/xml application/javascript;
gzip_vary on;
gzip_proxied any;

Compresses text responses to save bandwidth, at the cost of CPU cycles.

Expires Caching

location ~* \.(ico|jpe?g|gif|png|bmp|swf|flv)$ {
    expires 30d;
    access_log off;
}
location ~* \.(js|css)$ {
    expires 7d;
    access_log off;
}

Static assets are cached in browsers to reduce bandwidth and improve load times.

Anti‑Hotlinking

location ~* \.(jpg|gif|png|swf|flv|wma|wmv|asf|mp3|zip|rar)$ {
    valid_referers none blocked www.example.com example.com;
    if ($invalid_referer) { return 404; }
    access_log off;
}

Prevents other sites from directly linking to your resources.

Kernel Parameter Optimization

Adjust sysctl settings for high‑traffic servers, e.g.:

fs.file-max = 999999
net.ipv4.tcp_max_tw_buckets = 6000
net.ipv4.ip_local_port_range = 1024 65000
net.core.somaxconn = 40960
net.core.netdev_max_backlog = 262144
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_keepalive_time = 30

Apply changes with sysctl -p. These parameters increase the maximum number of open files, enlarge listen queues, and improve TCP handling.

System File Descriptor Limits

Linux defaults to 1024 open files. Increase limits in /etc/security/limits.conf:

* soft nofile 65535
* hard nofile 65535
* soft noproc 65535
* hard noproc 65535

Ensures the server can handle many simultaneous connections without hitting the "too many open files" error.

Summary

Properly tuning Nginx workers, CPU affinity, file limits, event model, fast file transfer, caching, compression, and underlying kernel parameters dramatically improves throughput, reduces latency, and enhances stability for high‑traffic web services.