Inside Nginx: Master/Worker Model, Async I/O, and Core Data Structures Explained

Daemon Process

Nginx starts as a daemon ("engine x"), a high‑performance HTTP and reverse‑proxy server that also supports IMAP/POP3/SMTP. After launch it runs in the background with a master process that manages several worker processes.

Master and Worker Processes

The master process handles signals, monitors worker status, and restarts workers that exit unexpectedly. Each worker process handles network events independently; workers are isolated, so a request is processed by only one worker.

The number of workers is usually set to match the CPU core count. Excess workers only increase context‑switch overhead. Nginx can bind workers to specific CPU cores to improve cache locality.

Thundering Herd Problem

All workers inherit the listening socket from the master. When a new connection arrives, every worker is notified, but only one can successfully accept() the socket; the others receive an error, which can cause load imbalance.

Why Processes Over Threads

Processes do not share memory, eliminating the need for locks and reducing overhead.

If a worker crashes, other workers continue serving requests, and the master quickly spawns a replacement.

Programming is simpler because there is no shared‑state concurrency.

Problems with Threads

Thread stacks consume more memory, and context switches are expensive. High‑concurrency workloads (e.g., thousands of simultaneous threads) can overwhelm the OS, leading to poor performance.

Asynchronous Non‑Blocking Model

Nginx uses an event‑driven, non‑blocking architecture: no threads are created per request, memory usage per connection is minimal, and there is no costly context switching.

According to Alibaba's Tengine team, a 24 GB machine can handle up to 2 million concurrent requests.

Connection Handling

Connections are defined in ngx_connection_t (found in src/core/ngx_connection.h/.c) and are TCP‑based. The workflow is:

Parse the configuration to obtain listening IP/port.

Master creates the listening socket and forks workers.

After the TCP three‑way handshake, a worker accept() s the socket and allocates a ngx_connection_t structure.

The worker then processes read/write events on that connection.

Nginx also supports a connection pool: each worker maintains a pre‑allocated array of ngx_connection_t objects (size = worker_connections) linked through free_connections. This avoids repeatedly allocating and freeing sockets.

Accept Mutex

To prevent multiple workers from competing for accept(), Nginx uses accept_mutex. The variable ngx_accept_disabled determines whether a worker should attempt to acquire the mutex; if the value is > 0, the worker skips accept() until the counter decrements.

HTTP Request Processing

An HTTP request is represented by ngx_http_request_t. The processing flow in a worker is:

Initialize the request object by reading client data.

Parse request headers.

Parse request body (if any).

Invoke the location‑specific handler.

Execute a series of phase handlers (e.g., rewrite, access, content).

Each phase handler typically:

Fetch the location configuration.

Generate an appropriate response.

Send response headers.

Send response body.

Keep‑Alive and Pipeline

Keep‑alive reuses a single TCP connection for multiple HTTP requests, reducing the number of TCP handshakes and TIME‑WAIT sockets. Pipeline builds on keep‑alive by allowing multiple requests to be sent without waiting for each response, improving throughput.

Lingering Close

When closing a connection, Nginx waits briefly (controlled by lingering_timeout) to read any remaining client data, preventing lost ACKs and ensuring graceful shutdown.

Core Data Structures

Arrays

ngx_array_t

is a dynamic array similar to a C++ vector, storing a pointer to elements, element count, element size, allocated capacity, and a memory pool.

Queues

ngx_queue_t

is a doubly‑linked list node with prev and next pointers.

Lists

ngx_list_t

implements a linked list of fixed‑size array parts ( ngx_list_part_t), allowing efficient traversal without per‑node allocation.

Strings

ngx_str_t

stores a length and a pointer to data; the string is not NUL‑terminated, which avoids copying.

Memory Pools

ngx_pool_t

(prototype ngx_pool_s) manages memory allocation for all the above structures, providing fast allocation and automatic cleanup.

Hash Tables

ngx_hash_t

is a static hash table built once during initialization; it uses a contiguous memory block for buckets, making lookups fast but disallowing insertions or deletions after creation.

Red‑Black Trees

ngx_rbtree_node_t

and ngx_rbtree_t implement a classic red‑black tree used for timers and other ordered data.

Illustrations