How I Optimized Nginx to Double My Site’s Concurrency

Problem Background

During peak traffic the Nginx reverse‑proxy reached a concurrency ceiling around 2,000 requests, with high response times, long‑tail latency, and a surge of 5xx errors. The default configuration ( worker_processes 1, worker_connections 1024, no gzip, no keep‑alive, etc.) was insufficient for the workload.

Baseline Measurements

The author captured pre‑tuning metrics using wrk and ab to record QPS, latency percentiles (P50/P90/P99), error rates, CPU, memory, socket states, and Nginx stub_status counters. Example command:

wrk -t8 -c200 -d30s --latency http://127.0.0.1:8080/

Typical output showed ~2,000 concurrent requests as the performance breakpoint.

Core Optimization Steps

Backup Configuration

TIMESTAMP=$(date +%Y%m%d-%H%M%S)
cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak${TIMESTAMP}
cp -a /etc/nginx/conf.d /etc/nginx/conf.d.bak${TIMESTAMP}

Worker Process Scaling Set the number of workers to match CPU cores automatically: worker_processes auto; Increase the file‑descriptor limit to avoid "Too many open files" errors: worker_rlimit_nofile 65535; Event Loop Tuning

events {
    worker_connections 4096;
    multi_accept on;
    use epoll;
}

This raises the maximum simultaneous connections and enables batch accept.

HTTP Block Enhancements

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    sendfile      on;
    tcp_nopush    on;
    tcp_nodelay   on;
    keepalive_timeout 65;
    keepalive_requests 1000;
    server_tokens off;
    # Gzip compression
    gzip on;
    gzip_min_length 1k;
    gzip_comp_level 5;
    gzip_types text/plain text/css application/json application/javascript image/svg+xml;
    gzip_vary on;
    # File descriptor cache for static assets
    open_file_cache max=10000 inactive=30s;
    open_file_cache_valid 60s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;
    # Rate limiting
    limit_req_zone $binary_remote_addr zone=req_zone:10m rate=30r/s;
    limit_conn_zone $binary_remote_addr zone=conn_zone:10m;
    limit_req_status 429;
    limit_conn_status 429;
    # Logging (JSON format)
    log_format main escape=json '{"time":"$time_iso8601","remote_addr":"$remote_addr","request":"$request","status":"$status","request_time":"$request_time","upstream_response_time":"$upstream_response_time"}';
    access_log /var/log/nginx/access.log main buffer=32k flush=5s;
    include /etc/nginx/conf.d/*.conf;
}

Upstream Keep‑Alive and Buffering

upstream backend {
    server 10.0.0.7:8080 max_fails=3 fail_timeout=10s;
    server 10.0.0.8:8080 max_fails=3 fail_timeout=10s;
    keepalive 64;   # Reuse connections to upstream
}

In the location / block:

location / {
    proxy_pass http://backend;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_connect_timeout 3s;
    proxy_send_timeout 30s;
    proxy_read_timeout 30s;
    proxy_buffer_size 16k;
    proxy_buffers 8 16k;
    proxy_busy_buffers_size 32k;
    proxy_buffering on;
}

Static Asset Caching

location /static/ {
    alias /var/www/static/;
    expires 7d;
    add_header Cache-Control "public, max-age=604800, immutable";
    access_log off;
}

location ~* \.(js|css|jpg|jpeg|png|gif|ico|svg|woff2?)$ {
    expires 7d;
    add_header Cache-Control "public, max-age=604800, immutable";
    access_log off;
    try_files $uri =404;
}

Client Request Limits

client_max_body_size 16m;
client_body_buffer_size 128k;
client_header_buffer_size 1k;
large_client_header_buffers 4 8k;
client_body_timeout 30s;
client_header_timeout 30s;
send_timeout 30s;
reset_timedout_connection on;

Testing After Each Change After applying a single configuration item, the author re‑ran the same wrk command and compared the key metrics (QPS, P99 latency, 5xx rate). The full test script is included in the article.

Verification and Monitoring

Key verification commands:

# Syntax check
nginx -t
# Reload safely
nginx -s reload
# Stub status
curl -s http://127.0.0.1:8080/nginx_status
# Baseline and post‑tuning wrk comparison
awk 'NR==13 || NR==27' wrk.before.txt wrk.after.txt

Additional monitoring includes top, iostat, ss -s, and custom jq parsing of JSON access logs to track $request_time vs $upstream_response_time.

Risk Warnings

Enabling all optimizations at once can hide the root cause of regressions. Apply changes incrementally.

Excessive worker_processes or worker_connections may exceed the OS file‑descriptor limit; adjust ulimit -n accordingly.

High gzip_comp_level (9) can saturate CPU without noticeable bandwidth gain; level 5 is a good compromise.

Setting keepalive_requests too high may keep idle connections open indefinitely.

Improper proxy_buffering on large responses can cause disk I/O spikes.

Rollback Procedure

# Restore backup
cp /etc/nginx/nginx.conf.bak${TIMESTAMP} /etc/nginx/nginx.conf
rm -rf /etc/nginx/conf.d
mv /etc/nginx/conf.d.bak${TIMESTAMP} /etc/nginx/conf.d
nginx -t && nginx -s reload

If the new configuration prevents Nginx from starting, use nginx -s stop, replace the config with the backup, and start Nginx again.

Production Checklist

Test all changes in a staging environment that mirrors production CPU, memory, and network.

Never use restart in production; prefer reload for zero‑downtime updates.

Match worker_processes to the actual core count (use auto).

Validate upstream health checks ( max_fails, fail_timeout) and keep‑alive pool size.

Monitor Active connections, Waiting, and 5xx percentages after deployment.

Ensure real‑IP headers are correctly set when behind a CDN or SLB.

Conclusion

Systematic Nginx tuning—starting from a solid baseline, adjusting worker and event settings, enabling efficient I/O (sendfile, tcp_nopush, tcp_nodelay), applying sensible gzip and caching, fine‑tuning proxy buffers and timeouts, and protecting the service with rate limits—can increase concurrent request capacity from ~2 k to over 6 k, reduce tail latency, and eliminate 5xx spikes. The article provides a repeatable, evidence‑based workflow that can be scripted and integrated into CI/CD pipelines for continuous performance improvement.