7 High‑Performance Server Event‑Handling Models: Reactor, Proactor and Beyond

Introduction

Server programs typically need to handle three kinds of events: I/O, signals, and timers. Two efficient event‑handling patterns are the Reactor and Proactor models. The Reactor pattern is usually implemented with synchronous I/O, while the Proactor pattern relies on asynchronous I/O.

In Linux, POSIX‑compatible asynchronous I/O functions (aio_read, aio_write) are only simulated in user space and work only for local files; they do not support sockets. Therefore, true Proactor implementations are unavailable on Linux, and the article demonstrates how to simulate Proactor behavior using synchronous I/O.

Both Reactor and Proactor are event‑dispatching network programming models. Reactor processes "pending" I/O events and requires the application to call read after the kernel signals readiness, so it is typically built on synchronous I/O. Proactor receives already‑completed I/O events from the kernel (often via signals), allowing the application to handle finished reads/writes without waiting, and is usually built on asynchronous I/O.

Why Simulate Proactor with Synchronous I/O?

Proactor can theoretically achieve higher efficiency because asynchronous I/O can overlap computation with I/O using DMA. However, Linux’s AIO implementation is incomplete, so most high‑concurrency network programs on Linux adopt the Reactor pattern (e.g., epoll) and simulate Proactor behavior.

Server Concurrency Models

Model 1: Single‑Threaded accept (no I/O multiplexing)

Analysis:

The main thread blocks on accept. When a client connects, accept creates a new socket descriptor (cfd) and the main thread processes all read/write operations serially.

If a new client connects while the main thread is handling an existing connection, the server cannot respond until the current processing finishes, limiting concurrency to one.

Pros:

Simple and clear socket programming flow; good for learning basic socket operations.

Cons:

Not a true concurrent model; handles only one request at a time (concurrency = 1).

Unsuitable for any production server.

Model 2: Single‑Threaded accept + Multi‑Threaded read/write (no I/O multiplexing)

Analysis:

The main thread blocks on accept. Upon a new connection, it creates a new thread to handle the client’s read/write business, then returns immediately to accept for more connections.

Each client is served by a dedicated thread, allowing concurrent handling of multiple clients.

Pros:

Improves concurrency compared to Model 1; each client gets its own thread, isolating business logic.

Flexible and easy to understand.

Cons:

Thread count grows linearly with client count, quickly hitting CPU and memory limits.

Long‑lived connections keep threads occupied, increasing resource consumption.

Still unsuitable for very high‑concurrency scenarios.

Model 3: Single‑Threaded I/O multiplexing

Analysis:

The main thread creates a listening socket (lfd) and uses an I/O multiplexing mechanism (select/epoll) to block‑wait for events.

When a client connects, the server accepts and adds the new socket (cfd) to the multiplexing set.

The same thread handles read/write for each ready socket, returning to the multiplexing loop after each operation.

If a client is performing I/O, the server cannot process other connections until the current operation finishes, leading to serial business processing.

Pros:

One thread can monitor many sockets (1:n), reducing thread overhead.

Blocking on the multiplexing call avoids busy‑waiting, improving CPU utilization.

Cons:

Only one client’s business logic runs at a time; other clients wait, causing latency under load.

Model 4: Single‑Threaded I/O multiplexing + Thread‑pool for business

Analysis:

Steps 1‑2 are identical to Model 3 (listen, accept, add cfd to multiplexing set).

When a socket becomes readable, the main thread reads the message and dispatches it to a pre‑started worker‑pool thread.

Worker threads process business logic only; the main thread later writes the response back to the client.

Pros:

Business processing is parallelized across the worker pool, reducing the time other clients wait for the main thread to finish business work.

Read/write still occurs in the main thread, so I/O concurrency remains 1, but overall throughput improves.

Cons:

Read/write channel is still single‑threaded; after business processing, responses still queue on the same I/O path.

Overall I/O performance is similar to Model 3; only business latency improves.

Model 5: Single‑Threaded I/O multiplexing + Thread‑pool for I/O (most common)

Analysis:

Before starting, the server creates a fixed‑size thread pool (N threads).

The main thread listens with epoll/select; when a new connection arrives, the accepted socket (cfd) is handed to one of the pool threads.

Each pool thread runs its own I/O multiplexing loop, monitoring the sockets assigned to it.

This allows many sockets to be monitored concurrently across multiple threads.

Pros:

Read/write concurrency scales with the number of pool threads; each thread can handle many sockets via multiplexing.

Theoretical maximum connections per process are N × (3 ~ 60 k) depending on epoll limits.

When N matches CPU core count, context‑switch overhead is minimized.

Cons:

Even though many sockets are monitored, each thread still processes I/O serially; sockets sharing a thread may experience latency.

Model 6: Multi‑process version of Model 5 (process pool)

Analysis:

Similar to Model 5, but the pool consists of separate processes instead of threads.

Each child process has its own listening sockets; the master process only creates the listening socket and distributes accept work via IPC.

Processes do not share memory, providing stronger isolation.

Pros:

Process isolation improves stability; a crash in one worker does not affect others (e.g., Nginx).

Cons:

Higher memory consumption compared to threads.

Model 7: Single‑Threaded I/O multiplexing + Thread‑pool for I/O + Per‑connection thread

Analysis:

Server creates a thread pool (N threads) before listening.

The main thread accepts connections and distributes each new socket to a pool thread for monitoring.

When a monitored socket becomes ready, the pool thread spawns a short‑lived child thread to perform the actual read/write work.

After the child thread finishes, the socket is returned to the pool thread for further monitoring.

Pros:

Combines the high I/O concurrency of Model 5 with per‑connection read/write parallelism, achieving a 1:1 ratio between client and read/write thread.

Maximum read/write channels equal the number of spawned child threads.

Cons:

Idealistic; requires a CPU core for each client thread, which is unrealistic on current hardware.

Creates significant context‑switch overhead and may degrade performance when client count is large.

Conclusion

Seven server concurrency structures are presented. Model 5 (thread‑pool I/O multiplexing) is the most widely used in high‑concurrency, high‑CPU‑utilization scenarios; Nginx follows a similar process‑pool design.

More complex models are not always better; they must be chosen based on hardware limits, CPU‑switch costs, and specific business requirements.