How to Scale Nginx for 500,000 Concurrent Connections: Architecture and Config Tips

1. Architecture and Horizontal Scaling

Deploy multiple Nginx instances as reverse‑proxy front‑ends and use a pool of application servers or micro‑service clusters behind them to avoid single‑point bottlenecks. Combine LVS, cloud load balancers, or DNS round‑robin with Nginx upstreams for traffic distribution, and employ containers or VMs with auto‑scaling policies to absorb traffic spikes.

2. OS and Hardware Preparation

Select high‑throughput network cards (10 Gbps or higher) and multi‑core CPUs, binding interrupts (IRQ affinity) to specific cores. Ensure ample memory and raise the file‑descriptor limit (e.g., ulimit -n) to support many simultaneous connections. Tune kernel parameters such as net.core.somaxconn, net.ipv4.ip_local_port_range, and net.ipv4.tcp_tw_reuse for optimal TCP handling.

3. Key Nginx Configuration Parameters

Critical directives include: worker_processes auto; – usually set to the number of CPU cores; each worker handles connections independently. worker_connections 102400; – maximum connections per worker. Theoretical max concurrency = worker_processes × worker_connections. For an 8‑core CPU with 100 000 connections per worker, the ceiling is roughly 800 000 concurrent connections. use epoll; – selects the epoll I/O multiplexing model for Linux. multi_accept on; – allows a worker to accept multiple connections per accept call, reducing accept overhead. worker_rlimit_nofile 200000; – raises the per‑process file‑descriptor limit.

4. Supporting Techniques and Optimizations

Further performance improvements:

TCP tuning: enable tcp_tw_reuse, lower tcp_fin_timeout, increase net.ipv4.tcp_max_syn_backlog.

Increase system file‑max via /proc/sys/fs/file-max and expand the local port range.

Offload static assets to a CDN or dedicated cache layer to reduce Nginx load.

Apply connection limiting, black/white lists, and rate‑limiting directives ( limit_conn, limit_req) to protect against abusive traffic.