Mastering Socket Programming: From Basics to Epoll and Reactor Patterns

Socket Programming

Network (socket) programming is divided into server‑side and client‑side development. On the server side, developers follow four steps: create a socket, bind it to an address and port, listen for incoming connections, and finally accept a connection request. On the client side, after creating a socket the developer simply calls connect to establish a connection to the server.

The three‑way TCP handshake proceeds as follows: the client sends a SYN packet, the server replies with SYN‑ACK, and the client acknowledges with ACK. After the handshake, the server places the new socket descriptor into the accept queue, ready for further processing.

Readable Socket Conditions

The socket's receive buffer contains at least the low‑water‑mark of data.

The connection’s read half is closed (FIN received).

A listening socket has pending connections (the completed‑connection count is non‑zero).

An error condition is pending; a read will return –1 and set errno accordingly.

Writable Socket Conditions

The socket's send buffer has at least the low‑water‑mark of free space and the socket is connected.

The connection’s write half is closed.

A non‑blocking connect has either succeeded or failed.

An error condition is pending on the socket.

An everyday analogy compares network blocking to waiting for a phone repair: waiting silently mirrors synchronous blocking, checking intermittently mirrors synchronous non‑blocking, receiving a callback after the repair mirrors asynchronous blocking, and multitasking while waiting mirrors I/O multiplexing or asynchronous non‑blocking.

When an application has few users, developers may use a multi‑process model for rapid development. The following diagram shows pseudo‑code for a synchronous blocking multi‑process server.

As traffic grows, each child process can monitor its own sockets, as illustrated by the subsequent pseudo‑code.

I/O Multiplexing and Reactor

When user count and business load increase, developers adopt I/O multiplexing, the Reactor pattern, and asynchronous non‑blocking techniques. The traditional select system call suffers from high memory overhead and a limited number of file descriptors.

Modern Linux systems favor epoll, which uses a red‑black tree to store registered descriptors and a ready list ( rdlist) that receives events directly, eliminating the need to poll every socket and greatly improving efficiency.

(1) OS Scheduling Principles

The kernel schedules processes in three states: running, blocked, and waiting. During the three‑way handshake, each packet (SYN, ACK) triggers a hardware interrupt, causing the current process to be pre‑empted and the socket to be placed into the socket queue.

The kernel then follows three steps to complete the handshake:

Network card receives the SYN packet, triggers an interrupt, and the kernel enqueues the socket into the socket queue.

If no pending connections exist, accept blocks and the socket is registered on the wait queue.

When a new connection arrives, the wait queue invokes a callback that moves the socket to the ready list ( rdlist).

When the ACK arrives, another interrupt moves the connection from the half‑open queue to the established queue, wakes the blocked accept process, and marks it ready for execution.

(2) Role of Epoll in Scheduling

Epoll monitors socket readability and writability. When creating an epoll instance, developers specify EPOLLIN and EPOLLOUT flags for the file descriptors they wish to watch. The epoll_wait call behaves like accept by notifying the application when events occur.

Typical usage converts socket creation and acceptance into epoll operations: the file descriptor is added to the epoll instance, along with the desired read/write events. When the listening descriptor signals an event, the application calls accept, marks the new descriptor as readable, and registers it with epoll for further monitoring.