How to Configure Nginx for a Million Connections – Full Template and Step‑by‑Step Guide

Applicable Scenarios & Prerequisites

This configuration is intended for high‑traffic web services that receive >1 million page views per day and must support >1 million concurrent TCP connections. The environment must meet the following minimums:

Operating System: RHEL/CentOS 8.0+ or Ubuntu 20.04 LTS+ (root or sudo rights to modify /etc/nginx/)

Linux kernel 4.15+ (kernel 5.10+ is recommended for BBR)

Nginx 1.20.x+ or 1.24.x+ (LTS releases)

CPU / RAM: at least 8 cores × 16 GB (recommended 16 cores × 32 GB); for ultra‑large scale 32 cores × 64 GB

Network: 10 Gbps+ NIC with support for tens of thousands of connections

Storage: SSD to avoid I/O bottlenecks

Expertise: senior ops engineer familiar with TCP/IP, HTTP, and performance tuning

Anti‑Pattern Warnings

Do not apply this configuration to low‑traffic services (<100 k PV/day); the default Nginx settings are sufficient.

Never tune parameters without a monitoring system – you cannot quantify the impact.

Avoid changing kernel or Nginx parameters during peak traffic; schedule changes during low‑traffic windows and observe for at least 30 minutes.

Do not blindly increase all limits (e.g., memory, file descriptors) beyond what the CPU can handle – excessive context switches will degrade performance.

This recipe is for native Nginx only; mixing with OpenResty, tengine, or other forks may cause syntax errors.

Environment & Version Matrix

OS: RHEL 8.5+ / CentOS Stream 9 or Ubuntu 20.04 LTS / 22.04 LTS (tested)

Kernel: 4.18.0‑305+ or 5.10+ (tested)

Nginx: 1.20.2 or 1.24.0 (tested)

Maximum theoretical connections: 1 million+

TLS: 1.2 / 1.3

OpenSSL: 1.1.1 or 3.0 (3.0 gives ~20‑30% performance gain)

Recommended hardware for production: 16 CPU × 32 GB RAM, 500 GB SSD; ultra‑large scale: 32 CPU × 64 GB RAM, 1 TB SSD

Quick Checklist

Verify Nginx version ≥ 1.20 ( nginx -v)

Confirm kernel can handle ≥1 million file descriptors ( ulimit -n ≥ 1000000)

Backup original configuration ( cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak)

Implementation Steps – Full Nginx Configuration

System‑Level Optimization (Step 1)

Increase the global file‑descriptor limit and tune TCP buffers. Append the following to /etc/sysctl.conf and apply with sysctl -p:

# File descriptor limits
fs.file-max = 10000000
# Listen queue depth
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
# Enable TIME_WAIT reuse
net.ipv4.tcp_tw_reuse = 1
# TCP timestamps (required for BBR)
net.ipv4.tcp_timestamps = 1
# Reduce FIN_WAIT2 timeout
net.ipv4.tcp_fin_timeout = 10
# Keepalive settings
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_keepalive_intvl = 15
# Socket buffers (128 MiB)
net.core.rmem_max = 134217728
net.core.wmem_max = 134217728
net.ipv4.tcp_rmem = 4096 87380 134217728
net.ipv4.tcp_wmem = 4096 65536 134217728
# NIC receive queue
net.core.netdev_max_backlog = 100000

Set per‑process limits for the Nginx user in /etc/security/limits.conf:

# Nginx user limits
nginx   soft    nofile  1000000
nginx   hard    nofile  1000000
nginx   soft    nproc   1000000
nginx   hard    nproc   1000000
# If you run Nginx as root (not recommended)
*       soft    nofile  1000000
*       hard    nofile  1000000
*       soft    nproc   1000000
*       hard    nproc   1000000

After editing, log out and back in (or restart the Nginx service) so the new limits take effect.

Nginx Configuration Optimization (Step 2)

Save the following template as /etc/nginx/nginx.conf. It has been validated on 2025‑11‑19.

# Nginx million‑scale configuration template v1.0
user nginx nginx;
worker_processes auto;
worker_priority -10;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
worker_rlimit_nofile 1000000;
worker_rlimit_core 0;

events {
    worker_connections 65536;   # 65k per worker (adjusted to system FD limits)
    use epoll;
    multi_accept on;
    accept_mutex off;           # improve concurrency on accept()
}

http {
    include /etc/nginx/mime.types;
    default_type application/octet-stream;
    log_format main '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" rt=$request_time uct=$upstream_connect_time uht=$upstream_header_time urt=$upstream_response_time';
    access_log /var/log/nginx/access.log main buffer=32k flush=5s;
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 60;
    client_body_timeout 10;
    client_header_timeout 10;
    send_timeout 10;
    client_max_body_size 20m;
    client_body_buffer_size 128k;
    client_header_buffer_size 1k;
    large_client_header_buffers 4 8k;
    gzip on;
    gzip_vary on;
    gzip_min_length 1k;
    gzip_comp_level 3;
    gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml+rss;
    gzip_disable "msie6";
    http2_max_field_size 16k;
    http2_max_header_size 32k;
    types_hash_max_size 2048;
    variables_hash_max_size 1024;
    variables_hash_bucket_size 128;
    server_names_hash_max_size 1024;
    server_names_hash_bucket_size 128;
    upstream_keepalive_connections 64;
    upstream_keepalive_timeout 60;
    upstream_keepalive_requests 100;
    proxy_buffering on;
    proxy_buffer_size 4k;
    proxy_buffers 8 4k;
    proxy_busy_buffers_size 8k;
    proxy_redirect off;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_connect_timeout 5s;
    proxy_send_timeout 10s;
    proxy_read_timeout 10s;
    limit_conn_zone $binary_remote_addr zone=conn_limit:10m;
    limit_conn conn_limit 10;
    limit_req_zone $binary_remote_addr zone=req_limit:10m rate=10r/s;
    limit_req zone=req_limit burst=20 nodelay;

    server {
        listen 8080;
        server_name _;
        location /health { access_log off; return 200 "ok
"; add_header Content-Type text/plain; }
        location /metrics { access_log off; return 200 "Nginx metrics endpoint
"; add_header Content-Type text/plain; }
    }

    server {
        listen 80 deferred reuseport backlog=65535;
        server_name _;
        root /var/www/html;
        index index.html index.htm;
        location ~* \.(jpg|jpeg|png|gif|ico|css|js|svg|woff|woff2|ttf)$ { expires 30d; add_header Cache-Control "public, immutable"; }
        location / { proxy_pass http://backend; proxy_http_version 1.1; proxy_connection_manager true; proxy_buffering on; }
        error_page 404 /404.html;
        error_page 500 502 503 504 /50x.html;
        location = /50x.html { root /usr/share/nginx/html; }
    }

    upstream backend {
        keepalive 64;
        server 192.168.1.10:8080 weight=1 max_fails=2 fail_timeout=10s;
        server 192.168.1.11:8080 weight=1 max_fails=2 fail_timeout=10s;
        server 192.168.1.12:8080 weight=1 max_fails=2 fail_timeout=10s;
        # Default load‑balancing method is round‑robin; alternatives: least_conn, ip_hash, etc.
    }
}

Validate the syntax and reload:

nginx -t
nginx -s reload

Performance Testing (Step 3)

Install a load‑testing tool (Apache Bench or wrk) and run a baseline test:

# Apache Bench example (1 M requests, 1 000 concurrent)
ab -n 1000000 -c 1000 http://localhost/
# wrk example (10 threads, 1 000 connections, 30 s)
wrk -t10 -c1000 -d30s http://localhost/

Expected results for a properly tuned 16 CPU × 32 GB server are >100 K RPS with average latency <100 ms and error rate ≈0%.

Core Processing Diagram

┌─────────────────────────────────────────┐
│      1 million TCP connections (mostly idle)      │
└──────────────────┬───────────────────────┘
                   │
          epoll_wait event queue
                   │
   ┌─────────────┴─────────────┐
   ▼                           ▼
Data arrives               Connection close
   │                           │
   └─────────────┬─────────────┘
                 │
          Nginx worker (single thread per worker)
   ┌─────────────┴─────────────┐
   ▼                           ▼
Read data (non‑blocking)   Send response (non‑blocking)

Key Parameters and Their Impact

worker_processes : set to the number of CPU cores (or auto). More workers than cores cause context‑switch overhead.

worker_connections : 65536 in the example; actual per‑worker limit = (system fs.file-max – reserved) / worker_processes. Adjust to keep total connections ≤ 1 million.

listen backlog : must match net.core.somaxconn (65535) to avoid “accept queue full”.

keepalive_timeout : 60 s is a good default; lower values increase reconnect overhead, higher values keep idle sockets longer.

tcp_nopush / tcp_nodelay : enable zero‑copy and disable Nagle’s algorithm for better throughput.

sendfile : on – uses kernel zero‑copy for static files.

proxy_buffering : on – buffers slow backend responses, preventing client‑side stalls.

Observability: Monitoring & Alerts

Native Nginx Stub Status

Add a dedicated server block (port 8080) with the following location:

location /nginx_status {
    stub_status on;
    access_log off;
    allow 127.0.0.1;
    deny all;
}

Query it with curl http://localhost:8080/nginx_status to obtain active connections, accepts, handled requests, and reading/writing/waiting counts.

Prometheus Exporter

Deploy the official exporter (GitHub URL: https://github.com/nginxinc/nginx-prometheus-exporter) and scrape the metrics endpoint. Example alert rules (compatible with Prometheus Alertmanager):

# Alert when active connections exceed 80% of the 1 million limit
expr: nginx_up * (nginx_connections_active / 1000000) > 0.8
for: 5m
annotations:
  summary: "Nginx active connections near limit"

# Alert when request rate exceeds 200 K RPS
expr: rate(nginx_http_requests_total[5m]) > 200000
for: 5m
annotations:
  summary: "Nginx request rate too high"

Common Issues & Troubleshooting

RPS stalls at 10‑20 K : Check CPU usage ( top). Increase worker_processes or upgrade hardware.

Many TIME_WAIT sockets : Enable net.ipv4.tcp_tw_reuse = 1 and consider reducing tcp_fin_timeout.

502 Bad Gateway : Verify backend health; increase upstream_keepalive_connections or the backend pool size.

Memory growth over time : Could indicate a leak in a third‑party module; upgrade Nginx or audit custom modules.

Change & Rollback Playbook

Gray‑scale Optimization

Deploy the new configuration to a staging environment and run a full load test for at least 2 hours.

Shift 10 % of production traffic to the new instance for 1 hour; monitor curl http://new-nginx:8080/nginx_status and error logs.

If metrics are stable, roll out to 100 % of traffic; continue watching with watch -n10 'curl -s http://localhost:8080/nginx_status'.

Rollback Conditions

Error rate > 0.1 %.

P95 latency increase > 50 %.

Frequent 502/503 responses.

Rollback command:

cp /etc/nginx/nginx.conf.bak /etc/nginx/nginx.conf
nginx -s reload

Best Practices

Run monthly load‑testing (e.g., wrk) to maintain a performance baseline.

Monitor active connections; alert if > 800 k (20 % headroom).

Analyze slow‑request logs (> 100 ms) regularly to spot regressions.

Automate deployment with Ansible/Terraform and keep configuration changes in Git (GitOps).

Reserve ~20 % headroom for CPU, memory, and connections to handle traffic spikes.

FAQ

Why do some claim Nginx can only handle 10 k connections?

Early defaults used worker_connections = 512, limiting total connections to a few thousand. Modern kernels support millions of file descriptors, and Nginx can be tuned to ~10‑20 k connections per worker. With multiple workers (e.g., 16) you can reach >1 million connections.

How much memory does 1 million connections consume?

Each connection typically uses 10‑30 KB (socket buffer + metadata). At ~20 KB per connection, 1 million connections require roughly 20 GB of RAM, which fits comfortably on a 32 CPU × 64 GB server.

What if the backend cannot keep up?

Enable proxy_buffering = on so Nginx buffers client requests while the backend processes them. However, the preferred solution is to scale the backend (add instances or optimise code).

What is a good keepalive_timeout ?

Long‑lived connections (WebSocket, gRPC) benefit from 300‑600 s. Typical HTTP APIs work well with 60‑120 s. Real‑time services may need 10‑30 s to avoid zombie connections.

Can Nginx replace a load balancer?

Technically yes, but high availability requires an external VIP (e.g., Keepalived) or a dedicated hardware/cloud LB in front of Nginx.

How should multiple Nginx instances be orchestrated?

For simple setups DNS round‑robin works, but production environments should use a soft LB (Keepalived VIP) or a cloud LB (ALB, SLB) to ensure seamless failover.