How Servers Serve Millions: Processes, Threads, and Event Loops Explained

Introduction

When you read this article you may wonder how a server delivers the page to you. The server receives a user request, fetches the article from a database and sends it back over the network.

Multi‑process Parallelism

The earliest and simplest way to handle many requests in parallel is to fork multiple processes. In Linux a parent process accepts connections and creates child processes with fork or exec to handle each request.

Advantages:

Simple programming and easy to understand.

Process isolation prevents a crash in one process from affecting others.

Fully utilizes multiple CPU cores.

Disadvantages:

Inter‑process communication (IPC) is required and can be complex and slow.

Creating and destroying processes incurs higher overhead than threads.

Multi‑thread Parallelism

Threads share the same address space, so communication is trivial and thread creation is lightweight. A thread can be created per request; if a thread blocks on I/O the others continue to run.

However, shared memory introduces problems such as race conditions, deadlocks, and synchronization bugs. Creating tens of thousands of threads also consumes significant memory and incurs scheduling overhead.

Event‑driven Programming

Beyond processes and threads, event‑driven (event‑based concurrency) handles many connections in a single thread by waiting for events, dispatching them to handler functions, and immediately returning to the event loop.

while(true) {
    event = getEvent();
    handler(event);
}

The event loop can process multiple requests because most of the time a request is waiting for I/O, which is delegated to the operating system.

I/O Multiplexing as Event Source

Linux treats everything as a file; sockets are file descriptors. I/O multiplexing (e.g., select, poll, epoll) monitors many descriptors and notifies when they become readable or writable, feeding events to the loop.

Blocking vs Non‑blocking I/O

Blocking (synchronous) I/O pauses the thread until the operation finishes, which stalls a single‑threaded event loop. Therefore an event‑driven server must avoid blocking calls.

Non‑blocking or asynchronous I/O returns immediately and lets the OS notify completion later, eliminating the stall.

Challenges of Event‑driven Design

Even with non‑blocking I/O, a single event loop cannot fully exploit multi‑core CPUs; spawning multiple loops reintroduces multithreading issues. Asynchronous code also splits logic between callers and callbacks, increasing complexity and maintenance cost.

Towards Better Solutions

To combine the simplicity of synchronous code with non‑blocking performance, user‑level threads (coroutines) are introduced, allowing many lightweight execution contexts on top of an event loop.

Conclusion

High‑concurrency techniques have evolved from multi‑process to multi‑thread to event‑driven architectures, each with trade‑offs. Understanding this history helps grasp modern server designs.