Ultimate Guide to Designing Nginx Architecture for 1 Million QPS

Why Nginx Handles High Concurrency

Nginx can sustain millions of connections thanks to three core mechanisms:

Master‑Worker Process Model – a single master manages multiple independent workers.

Reactor Event Model – each worker uses epoll to handle tens of thousands of connections on a single thread.

Non‑Blocking I/O – reads are performed only when data is ready, avoiding blocking calls.

Master‑Worker Diagram

Master
├── Worker
├── Worker
├── Worker
└── Worker

Reactor Workflow

Event registration
   ↓
Event notification
   ↓
Request processing

Non‑Blocking I/O

Traditional: read() blocks
Nginx: epoll + non‑blocking
Process only when data is ready

Million‑QPS Architecture Design

A single machine cannot sustain one million QPS; a layered architecture is required:

DNS/GSLB – geographic routing

LVS (Layer‑4 load balancer)

Nginx gateway cluster (Layer‑7)

Service cluster – business logic

Cache – Redis/CDN

Database

Traffic Reduction per Layer

Assuming 1 000 000 QPS at the entry point, the layered design reduces traffic as follows:

DNS: 1 000 000 QPS

LVS: 1 000 000 QPS

Nginx: 700 000 QPS

Service: 300 000 QPS

Database: 10 000 QPS

Linux Kernel Optimizations

Key sysctl parameters to lift kernel limits:

net.core.somaxconn = 65535

net.ipv4.tcp_max_syn_backlog = 65535

net.ipv4.ip_local_port_range = 1024 65535

net.ipv4.tcp_tw_reuse = 1

net.ipv4.tcp_fin_timeout = 15

fs.file-max = 6815744

net.netfilter.nf_conntrack_max = 1048576

Apply with sysctl -p.

File‑Descriptor Limits

/etc/security/limits.conf
* soft nofile 1048576
* hard nofile 1048576

Verify with ulimit -n.

Nginx Core Tuning

Main Configuration

user nginx;
worker_processes auto;
worker_rlimit_nofile 1000000;

Events Block

events {
    use epoll;
    worker_connections 65535;
    multi_accept on;
    accept_mutex off;
}

Theoretical maximum connections: worker_processes * worker_connections. Example: 16 CPU × 65535 ≈ 1 000 000 connections.

HTTP Block

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 30;
    keepalive_requests 10000;
    client_body_buffer_size 128k;
    client_header_buffer_size 4k;
    gzip on;
    gzip_comp_level 2;
    access_log off;
}

Zero‑Copy Optimization

Enable sendfile on and sendfile_max_chunk 1m to reduce copy operations from four to two.

CPU and NIC Optimizations

When QPS exceeds 300 k, bottlenecks move to NIC, CPU, and NUMA. Commands:

Check NIC queues: ethtool -l eth0 Enable multi‑queue: ethtool -L eth0 combined 16 Bind workers to CPUs:

worker_cpu_affinity auto;

Rate‑Limiting Protection

IP‑Based Limiting

limit_req_zone $binary_remote_addr zone=req_limit:10m rate=100r/s;
server {
    location / {
        limit_req zone=req_limit burst=200;
        proxy_pass http://backend;
    }
}

Connection Limiting

limit_conn_zone $binary_remote_addr zone=conn_limit:10m;
limit_conn conn_limit 100;

OpenResty Lua Dynamic Limiting

lua_shared_dict conn_limit_store 10m;
location / {
    access_by_lua_block {
        local ip = ngx.var.binary_remote_addr
        -- custom logic here
    }
    proxy_pass http://backend;
}

Load Testing

Recommended tools: wrk, wrk2.

Test Command

wrk -t64 -c20000 -d60s http://nginx/test

Parameters

-t

: threads -c: connections -d: duration

Sample Results

200 k QPS – 40% CPU – 2 ms latency

400 k QPS – 68% CPU – 3 ms latency

600 k QPS – 82% CPU – 5 ms latency

800 k QPS – 95% CPU – 8 ms latency

Single‑machine ceiling observed at ~900 k QPS; keep CPU below 50 % in production.

Real‑World Incident

During a major e‑commerce promotion, QPS hit 100 k and Nginx CPU spiked to 100 % because accept_mutex was left on, causing worker lock contention. Fix by setting accept_mutex off and multi_accept on, which raised throughput from 80 k to 180 k QPS.

Full Enterprise‑Grade Configuration

user nginx;
worker_processes auto;
worker_rlimit_nofile 1000000;

events {
    use epoll;
    worker_connections 65535;
    multi_accept on;
    accept_mutex off;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 30;
    keepalive_requests 10000;
    gzip on;
    gzip_comp_level 2;
    access_log off;

    upstream backend {
        least_conn;
        keepalive 300;
        server 10.0.0.1:8080;
        server 10.0.0.2:8080;
    }

    server {
        listen 80 backlog=65535;
        location / {
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            proxy_pass http://backend;
        }
    }
}

Core Principles for Million‑QPS Systems

Layered Architecture : DNS → LVS → Nginx → Service

Stateless Design : enables horizontal scaling

Cache First : push traffic to Redis/CDN before DB

Rate‑Limiting Protection : prevents avalanche failures

Final Takeaway

Architecture sets the upper bound; tuning defines the lower bound.