Master Nginx: From Source Compilation to Advanced Performance Tuning

Installation

Nginx is widely adopted for its excellent performance, stability, simple configuration, and cross‑platform support. To achieve a lean installation, compile it from source and selectively include only the needed modules.

1. Minimal modules

Using package managers (rpm, deb) often installs many optional modules that increase the binary size and surface area for security issues. Compiling from source allows you to exclude unnecessary modules such as mail, uwsgi, memcache, etc.

./configure \
 "--prefix=/App/nginx" \
 "--with-http_stub_status_module" \
 "--without-http_auth_basic_module" \
 "--without-http_autoindex_module" \
 "--without-http_browser_module" \
 "--without-http_empty_gif_module" \
 "--without-http_geo_module" \
 "--without-http_limit_conn_module" \
 "--without-http_limit_req_module" \
 "--without-http_map_module" \
 "--without-http_memcached_module" \
 "--without-http_proxy_module" \
 "--without-http_referer_module" \
 "--without-http_scgi_module" \
 "--without-http_split_clients_module" \
 "--without-http_ssi_module" \
 "--without-http_upstream_ip_hash_module" \
 "--without-http_upstream_keepalive_module" \
 "--without-http_upstream_least_conn_module" \
 "--without-http_userid_module" \
 "--without-http_uwsgi_module" \
 "--without-mail_imap_module" \
 "--without-mail_pop3_module" \
 "--without-mail_smtp_module" \
 "--without-poll_module" \
 "--without-select_module" \
 "--with-cc-opt='-O2'"

Adjust the list according to the features your site actually uses. For example, keep the --with-http_stub_status_module for status monitoring and the FastCGI module for PHP.

2. GCC optimization (optional)

GCC provides several optimization levels:

-O0 : No optimization.

-O / -O1 : Small size and speed improvements without significantly increasing compile time.

-O2 : Includes -O1 optimizations and adds more without loop unrolling or function inlining; recommended for most software.

-Os : Optimizes for smallest binary size.

-O3 : Enables aggressive optimizations such as inlining and loop unrolling; may increase compile time and binary size without noticeable runtime gains.

For Nginx and most other programs compiled with GCC, using -O2 is a safe and effective choice.

Configuration

Performance tuning focuses on efficient use of CPU, memory, disk I/O, and network I/O. Below are key directives to adjust in nginx.conf:

1. worker_processes

Defines the number of worker processes. Set it to the number of CPU cores or use auto for automatic detection.

2. worker_cpu_affinity

Binds each worker to a specific CPU core to achieve a more balanced CPU utilization.

worker_processes    4;
worker_cpu_affinity 0001 0010 0100 1000;

3. worker_rlimit_nofile

Sets the maximum number of open files per worker. It should be at least the system limit divided by the number of workers.

4. accept_mutex

When set to on, Nginx wakes one worker at a time to accept new connections, reducing the “thundering herd” problem. Setting it to off can increase throughput but may raise context‑switch overhead.

5. use

Specifies the event model. On Linux 2.6+ the recommended value is epoll; on FreeBSD use kqueue.

6. worker_connections

Maximum simultaneous connections per worker. It should not exceed worker_rlimit_nofile.

7. open_file_cache

Enables caching of file descriptors to avoid repeated open/close operations.

open_file_cache max=65536 inactive=60s;

8. Logging

Heavy logging can degrade performance. Consider disabling access logs in high‑traffic environments or writing them to a tmpfs, and set the error log level to error or crit.

9. server_tokens

Set to off to hide the Nginx version number in response headers, improving security.

10. gzip

Enable gzip compression (recommended). Use a moderate compression level (1‑4) to balance CPU usage and bandwidth savings.

gzip on;
gzip_comp_level 2;
gzip_min_length 1k;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml+rss text/javascript;

11. expires

Configure browser caching for static assets to reduce repeated downloads.

location ~ .+\.(gif|jpg|jpeg|png|bmp|swf)$ {
    expires 30d;
}
location ~ .+\.(js|css|xml|javascript|txt|csv)$ {
    expires 30d;
}

12. keepalive_timeout

Enables HTTP persistent connections. Set a positive value for static‑heavy sites; set to 0 for purely dynamic content.

Kernel tuning

Default Linux kernel parameters are not optimal for high concurrency. Adjust them via /etc/sysctl.conf (persistent) or the /proc filesystem (temporary). Example configuration:

net.core.rmem_default = 262144
net.core.rmem_max = 16777216
net.core.wmem_default = 262144
net.core.wmem_max = 16777216
net.core.somaxconn = 262144
net.core.netdev_max_backlog = 262144
net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_max_tw_buckets = 10000
net.ipv4.ip_local_port_range = 1024 65500
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_synack_retries = 1
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_mem = 786432 1048576 1572864
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.sem = 250 32000 100 128
vm.swappiness = 10

Apply changes with sysctl -p.

Architecture

Nginx excels at serving static files and proxying requests, supporting layer‑7 load balancing and fault isolation. For large‑scale sites, separate static content onto dedicated domains or subdomains, and consider front‑end load balancers (LVS, F5) or caching layers (Varnish, Squid). Newer Nginx versions can directly read/write Memcache, reducing load on backend application servers.