How to Eliminate Nginx’s Thundering Herd Problem: Accept & Epoll Solutions

Nginx is a high‑performance web and reverse‑proxy server widely used in high‑concurrency scenarios. In its multi‑process model, it can suffer from the thundering herd problem, where many processes are awakened for a single event but only one can handle it, causing resource waste and context‑switch overhead.

1. Nginx Multi‑process Model and Herd Problems

Nginx uses the classic Master‑Worker multi‑process model:

Master process : manages the lifecycle of worker processes, loads configuration files, and listens on ports.

Worker process : independently handles client requests using an asynchronous event‑driven model based on epoll/kqueue.

All worker processes inherit the same listening socket (via bind and listen system calls). When a new client connection arrives, every worker is awakened, leading to a typical thundering herd situation.

2. Accept Herd

2.1 Problem Definition

Accept herd occurs when multiple processes listen on the same listening socket ( listen_fd). When a new connection arrives, all processes are awakened, but only one can successfully execute accept() and receive the connection; the others receive errors such as EAGAIN or ECONNABORTED and go back to sleep, wasting resources.

2.2 Causes

Shared listening socket : Workers inherit the master’s listen_fd and call accept() directly.

Non‑atomic accept() : In early Linux kernels, accept() is not atomic, so multiple processes may be awakened simultaneously.

2.3 Solutions

(1) Linux Kernel Improvements

Kernel‑level optimization : Since Linux 2.6.x, the kernel introduces wait queues and the exclusive flag WQ_FLAG_EXCLUSIVE, ensuring only the first waiting process is awakened when a new connection arrives.

Atomic accept() : The kernel guarantees the atomicity of accept(), allowing only one process to succeed.

(2) Nginx Shared‑Lock Mechanism

Even after kernel optimizations, Nginx still uses a shared lock ( accept_mutex) to control accept herd:

Lock competition flow :

Each worker tries to acquire the shared lock ( ngx_shmtx_t).

The worker that obtains the lock listens on listen_fd and processes new connections; others skip the listening stage.

After releasing the lock, other workers compete again, achieving dynamic load balancing.

Code implementation :

typedef struct {
    ngx_atomic_t *lock;       // atomic lock pointer
    ngx_atomic_t *wait;       // wait counter
    sem_t        sem;         // POSIX semaphore
    ngx_uint_t   spin;        // spin count control
} ngx_shmtx_t;

void ngx_shmtx_lock(ngx_shmtx_t *mtx);
ngx_atomic_t ngx_shmtx_trylock(ngx_shmtx_t *mtx);

Dynamic load balancing : The variable ngx_accept_disabled controls a worker’s accept frequency. When a worker reaches a connection threshold (e.g., 7/8 of the maximum), it temporarily stops competing for the lock, preventing resource skew.

(3) SO_REUSEPORT Feature (Modern Kernels)

Kernel‑level load balancing : Linux 3.9+ introduces SO_REUSEPORT, allowing multiple processes to bind to the same port. The kernel automatically distributes new connections among them, eliminating the need for user‑space lock competition.

Nginx configuration :

events {
    use epoll;
}
http {
    listen 80 reuseport;
    accept_mutex off;
}

Enabling reuseport makes the kernel assign connections directly, while disabling accept_mutex reduces user‑space overhead.

3. Epoll Herd

3.1 Problem Definition

Epoll herd occurs when multiple processes call epoll_wait() on the same listening socket ( listen_fd). When a new connection arrives, all processes are awakened, but only one can successfully handle the connection, similar to accept herd but rooted in the epoll mechanism.

3.2 Causes

Shared epoll instances : Each worker creates its own epoll instance via epoll_create() and adds listen_fd to it.

Non‑atomic epoll_wait() : In early Linux versions, epoll_wait() is not atomic, allowing multiple processes to be awakened simultaneously.

3.3 Solutions

(1) Linux Kernel Improvements

Kernel‑level optimization : Since Linux 2.6.18+, the epoll mechanism incorporates similar atomic wake‑up logic as accept, waking only the first waiting process.

Exclusive flag : The kernel adds WQ_FLAG_EXCLUSIVE to the epoll wait queue, ensuring only one process is notified.

(2) Nginx accept_mutex Mechanism

Nginx further avoids epoll herd by using the shared lock:

Lock controls listening : Only the process holding accept_mutex adds listen_fd to its epoll instance.

Unlock processes skip listening : Processes without the lock do not listen, preventing their epoll_wait() from being awakened.

Dynamic listening management : The lock‑holding process keeps listen_fd in epoll until it releases the lock, ensuring no connection loss.

(3) SO_REUSEPORT Feature (Modern Kernels)

Alternative to accept_mutex : Enabling reuseport lets the kernel distribute connections among processes, completely eliminating epoll herd.

Performance benefits : Removes user‑space lock overhead and improves multi‑core CPU utilization, especially in high‑concurrency scenarios.

4. Comparison and Summary of the Two Herd Types

Both accept and epoll herd stem from multiple processes sharing the same listening resources and non‑atomic kernel operations. Kernel improvements (wait queues, exclusive flags) mitigate the issue, while Nginx’s shared‑lock mechanisms and the SO_REUSEPORT feature provide practical solutions. For modern kernels, SO_REUSEPORT is the preferred approach; for older kernels, the accept_mutex strategy remains compatible.

5. Conclusion

Nginx resolves both accept and epoll herd problems through its shared‑lock mechanism ( accept_mutex) and modern kernel features like SO_REUSEPORT. Traditional user‑space locking is more complex but offers broader compatibility, whereas SO_REUSEPORT leverages kernel‑level load balancing to minimize overhead and maximize multi‑core performance. In production, it is recommended to enable reuseport and disable accept_mutex for high‑performance environments, while falling back to accept_mutex for older kernel versions.