Boost Nginx to 100k+ Concurrent Connections: Essential Linux Kernel Tweaks

Why kernel optimization is needed?

Linux defaults are designed for desktop use, limiting concurrent connections to about 1024. Nginx can handle tens of thousands, but the kernel restrictions prevent it from reaching its potential.

Optimization essence: remove limits, improve network stack processing, avoid resource exhaustion.

8 essential kernel parameters (with details)

Configuration file: /etc/sysctl.conf – apply changes with sysctl -p .

1️⃣ Increase local port range (fix "Cannot assign requested address")

net.ipv4.ip_local_port_range = 1024 65535
# default: 32768 60999 (≈28k ports)
# after tuning: >60k ports, supports more outbound connections

2️⃣ Reuse TIME-WAIT connections (avoid port exhaustion)

net.ipv4.tcp_tw_reuse = 1
# Allows reusing TIME-WAIT sockets for new requests
# Do NOT set tcp_tw_recycle=1 (deprecated, breaks NAT)

3️⃣ Reduce FIN timeout (speed up connection release)

net.ipv4.tcp_fin_timeout = 30
# default 60s, reduced to 30s for faster resource release
# suitable for short‑lived high‑frequency connections

4️⃣ Increase connection queues (prevent accept() failed)

net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
# somaxconn: completed‑handshake queue length (Nginx listen backlog cannot exceed this)
# tcp_max_syn_backlog: half‑open SYN queue length

Adjust Nginx accordingly, e.g.,

listen 80 backlog=65535;

5️⃣ Raise file descriptor limit (solve "Too many open files")

# /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
root soft nofile 65536
root hard nofile 65536

Each TCP connection consumes one file descriptor; default 1024 is quickly exhausted under high load. Changes require re‑login or Nginx restart.

6️⃣ Optimize TCP buffers (increase throughput)

net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216

Dynamic TCP read/write buffers adapt to high BDP networks, preventing retransmissions and packet loss.

7️⃣ Enable TCP Fast Open (optional, speeds first connection)

net.ipv4.tcp_fastopen = 3

Allows data in SYN packet, reducing one RTT; works for HTTPS front‑page optimization when both client and server support it (Linux 3.7+).

8️⃣ Disable TCP SACK (optional in extreme loss scenarios)

net.ipv4.tcp_sack = 0

SACK can increase CPU load under high loss; keep enabled (default 1) unless a clear bottleneck is identified.

How to verify after tuning?

Check that parameters are effective, e.g., sysctl net.core.somaxconn, sysctl net.ipv4.ip_local_port_range.

View current connections, e.g., ss -s or netstat -an | wc -l.

Run load tests (wrk, ab) and compare error rate, QPS, latency.

Safety and stability recommendations

Avoid blindly increasing all parameters – may cause memory exhaustion and OOM kills.

Synchronize Nginx worker_connections with ulimit -n.

When using VMs/containers, ensure host limits allow the changes.

Recommended configuration for a 16 GB server

# /etc/sysctl.conf
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 30
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216

# /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536

# nginx.conf
events {
    worker_connections 65535;
    use epoll;
    multi_accept on;
}

Summary: Three‑step high‑concurrency optimization

1. Lift limits: port range, file descriptors, connection queues.
2. Boost efficiency: TIME‑WAIT reuse, TCP buffer tuning.
3. Validate with benchmarks: let data speak, avoid “magical” tuning.