Master Nginx: From Source Build to High‑Performance Tuning

Installation

Nginx is widely adopted for its performance, stability, simple configuration, and cross‑platform support, becoming the second‑largest web server after Apache. Many existing guides only cover editing the configuration file and tweaking OS kernel parameters, lacking systematic optimization.

We start with compiling Nginx from source, allowing us to exclude unnecessary modules and apply custom build options. A typical configure command is:

./configure \ "--prefix=/App/nginx" \ "--with-http_stub_status_module" \ "--without-http_auth_basic_module" \ "--without-http_autoindex_module" \ "--without-http_browser_module" \ "--without-http_empty_gif_module" \ "--without-http_geo_module" \ "--without-http_limit_conn_module" \ "--without-http_limit_req_module" \ "--without-http_map_module" \ "--without-http_memcached_module" \ "--without-http_proxy_module" \ "--without-http_referer_module" \ "--without-http_scgi_module" \ "--without-http_split_clients_module" \ "--without-http_ssi_module" \ "--without-http_upstream_ip_hash_module" \ "--without-http_upstream_keepalive_module" \ "--without-http_upstream_least_conn_module" \ "--without-http_userid_module" \ "--without-http_uwsgi_module" \ "--without-mail_imap_module" \ "--without-mail_pop3_module" \ "--without-mail_smtp_module" \ "--without-poll_module" \ "--without-select_module" \ "--with-cc-opt='-O2'"

Adjust the list of modules according to actual needs; for example, remove the SSI module if not required. Use ./configure --help to view all options.

GCC Optimization (Optional)

GCC provides five optimization levels:

-O0 : No optimization.

-O / -O1 : Small code size and modest speed gains.

-O2 : Balanced optimizations without loop unrolling or function inlining.

-Os : Optimizes for smallest size (similar to -O2.5).

-O3 : Enables all -O2 optimizations plus aggressive inlining and loop transformations; rarely needed for Nginx.

For most software, including Nginx, -O2 is the recommended safe choice.

Configuration Tuning

Performance hinges on proper use of CPU, memory, disk I/O, and network I/O. Key directives in nginx.conf include:

worker_processes : Set to the number of CPU cores (or auto) to match available processing power.

worker_cpu_affinity : Bind workers to specific CPU cores to achieve balanced utilization.

worker_rlimit_nofile : Raise the per‑process open‑file limit; typically set to the system’s maximum open files divided by the number of workers.

accept_mutex : on avoids the “thundering herd” problem; off can increase throughput at the cost of more context switches.

use : Choose the most efficient I/O model (e.g., epoll on Linux 2.6+).

worker_connections : Maximum simultaneous connections per worker; should not exceed worker_rlimit_nofile.

open_file_cache : Enable caching of frequently accessed files to reduce file‑system overhead. Example: open_file_cache max=65536 inactive=60s; access_log / error_log : Disable or reduce logging in high‑traffic environments, or write logs to a tmpfs to lessen disk I/O.

server_tokens : Set to off to hide Nginx version information for security.

gzip : Enable compression (typically gzip on; with gzip_comp_level 2) to save bandwidth; exclude already compressed assets.

expires : Configure browser caching for static assets, e.g., expires 30d; for images, CSS, JS.

keepalive_timeout : Enable persistent connections for static‑heavy sites; disable for highly dynamic content.

Sample Configuration Snippet

user nginx nginx; worker_processes auto; error_log logs/error.log error; pid logs/nginx.pid; worker_rlimit_nofile 65536; events { use epoll; accept_mutex off; worker_connections 65536; } http { include mime.types; default_type text/html; charset UTF-8; open_file_cache max=65536 inactive=60s; open_file_cache_valid 80s; open_file_cache_min_uses 1; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log logs/access.log main; sendfile on; server_tokens off; gzip on; gzip_comp_level 2; gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript; keepalive_timeout 60; server { listen 80; server_name localhost; root /App/web; location ~ \.php$ { fastcgi_pass unix:/tmp/php.sock; fastcgi_index index.php; include fastcgi.conf; fastcgi_cache cache_fastcgi; } } }

Kernel Parameter Tuning

Default Linux kernel settings are not optimal for high‑concurrency workloads. Adjust /etc/sysctl.conf (or use /proc/sys for temporary changes) with values such as:

net.core.rmem_default = 262144 net.core.rmem_max = 16777216 net.core.wmem_default = 262144 net.core.wmem_max = 16777216 net.core.somaxconn = 262144 net.core.netdev_max_backlog = 262144 net.ipv4.tcp_max_orphans = 262144 net.ipv4.tcp_max_syn_backlog = 262144 net.ipv4.tcp_max_tw_buckets = 10000 net.ipv4.ip_local_port_range = 1024 65500 net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_syncookies = 1 net.ipv4.tcp_fin_timeout = 30 net.ipv4.tcp_keepalive_time = 600 net.ipv4.tcp_keepalive_intvl = 30 net.ipv4.tcp_keepalive_probes = 3 net.ipv4.tcp_mem = 786432 1048576 1572864 fs.file-max = 6815744 vm.swappiness = 10

Apply changes with sysctl -p.

Architecture Recommendations

Nginx excels at serving static files and proxying requests. For large sites, separate static content onto dedicated domains or subdomains, possibly behind a CDN. If a single Nginx instance cannot handle the load, place a hardware or software load balancer (LVS, F5) in front of multiple Nginx nodes, and consider adding Varnish or Squid for additional caching.

Newer Nginx versions can compile with native Memcached support, reducing the need to involve backend application servers for cache lookups.