Eliminate Select’s High CPU Usage: A Step‑by‑Step Epoll Source Dive in Pure C

1. Warm‑up: Socket Basics

Before diving into epoll, the article reviews the socket API. It explains that a socket is an endpoint linking an application to the network stack, describes creating a TCP socket with socket(AF_INET, SOCK_STREAM, 0), binding it to an address and port using bind(), listening with listen(), and accepting connections via accept(). These steps lay the groundwork for later epoll usage.

2. Why select/poll Become Performance Bottlenecks

Select and poll allow monitoring multiple file descriptors, but both scan the entire fd set linearly (O(n)) on each call. When the number of connections grows to thousands, the kernel must copy the fd set between user and kernel space and traverse every descriptor, causing noticeable CPU spikes and latency. The article illustrates this with an analogy of asking each employee whether they are free before assigning a task.

3. Epoll: Design and Advantages

Epoll replaces the linear scan with an event‑driven model. The kernel maintains a red‑black tree of registered fds and a ready‑list. When a descriptor becomes readable or writable, the kernel adds it to the ready‑list, so epoll_wait() only iterates over active fds, reducing overhead to O(log N) for registration and O(k) for waiting, where k is the number of ready events.

The article also mentions epoll’s use of shared memory (mmap) to avoid extra data copies between kernel and user space, further improving throughput.

3.1 Core Epoll Functions

epoll_create1(int flags) – creates an epoll instance; flags can be 0 or EPOLL_CLOEXEC.

epoll_ctl(int epfd, int op, int fd, struct epoll_event *event) – registers, modifies, or removes a fd. op may be EPOLL_CTL_ADD, EPOLL_CTL_MOD, or EPOLL_CTL_DEL. The event structure contains a bitmask of events (e.g., EPOLLIN, EPOLLOUT, EPOLLET) and user data.

epoll_wait(int epfd, struct epoll_event *events, int maxevents, int timeout) – blocks until at least one registered fd becomes ready, filling the events array with up to maxevents entries.

3.2 Level‑Triggered (LT) vs Edge‑Triggered (ET)

LT is the default mode: as long as a fd is readable or writable, epoll_wait() will keep reporting the event on each call. This is simple and works well for modest connection counts.

ET reports an event only when the fd’s state changes (e.g., from not readable to readable). After receiving an ET notification, the application must read or write until read() returns EAGAIN / EWOULDBLOCK to avoid missing further data. ET requires non‑blocking sockets and is suited for high‑concurrency servers.

4. Full Epoll Server Implementation in Pure C

The article provides a complete, compilable example that demonstrates a high‑performance TCP server using epoll. The code follows these steps:

Create the listening socket (TCP, IPv4) and bind to PORT 8888.

Listen with a backlog of 10.

Create an epoll instance with epoll_create1(0).

Register the listening socket for EPOLLIN events using epoll_ctl(..., EPOLL_CTL_ADD, ...).

Enter the event loop :

Call epoll_wait() (blocking indefinitely).

If the ready fd is the listening socket, accept() the new connection, set it to non‑blocking mode, and register it for EPOLLIN | EPOLLET.

Otherwise, the fd belongs to a client. In ET mode, repeatedly read() into a buffer until EAGAIN or EWOULDBLOCK. On successful reads, the server prints the data and sends a fixed acknowledgment back to the client.

Handle EOF (read returns 0) by closing the client socket, and handle other errors by logging with perror() and closing the socket.

Cleanup – close the listening socket and the epoll file descriptor before exiting.

Compilation and execution are straightforward:

gcc -o epoll_server epoll_server.c
./epoll_server

The article encourages readers to run the program, observe the interaction between server and client, and modify the code to suit different business scenarios.

5. Practical Takeaways

Use epoll instead of select/poll for large numbers of concurrent connections to avoid O(n) scanning overhead.

Prefer edge‑triggered mode with non‑blocking sockets for high‑throughput services, but be aware of the need to drain the socket completely on each notification.

The provided source code can serve as a template for building embedded gateways, IoT back‑ends, or any Linux‑based high‑concurrency server.