Eliminate Nginx Thundering Herd: Mutex, ReusePort, Multi_Accept & KeepAlive

Nginx, as a core component of large‑scale architectures, can suffer from the thundering herd effect, where multiple workers wake up simultaneously to accept a new connection but only one succeeds, causing the others to immediately go back to sleep.

Why the Thundering Herd Happens

The root cause is that all workers listen on the same port and compete for the accept call. In high‑concurrency short‑connection scenarios, new connections arrive frequently, waking many workers, yet only one can actually accept the connection.

Solutions

1️⃣ accept_mutex (general solution)

Enable a mutex so that only one worker can call accept at a time. accept_mutex on; 2️⃣ reuseport (optimal solution)

Configure each worker to listen on the same port with the reuseport flag, letting the kernel distribute connections and completely avoid competition. listen 80 reuseport; 3️⃣ multi_accept (auxiliary optimization)

Allow a worker to accept multiple connections in a single call, reducing the number of wake‑ups. multi_accept on; 4️⃣ Reduce connection establishment (fundamental solution)

Enable keep‑alive and upstream connection reuse to minimize the frequency of new connections. keepalive_timeout 60; These configurations together mitigate the thundering herd effect, improving Nginx’s efficiency under high load.