How Tomcat Implements NIO: A Deep Dive into I/O Multiplexing and NioEndpoint

I/O Multiplexing Model Overview

Tomcat's NIO implementation relies on I/O multiplexing. The article reviews the classic I/O models—blocking, non‑blocking, select/poll/epoll, signal‑driven, and asynchronous—highlighting that the focus here is on the multiplexing model, which uses a single thread (or a few threads) to manage many connections.

Tomcat's Support for I/O Models

Since Tomcat 6, the server supports the NIO model via the java.nio package. The article notes that both request and response bodies are handled in a blocking manner, with the underlying NIO selector used for timeout control rather than pure non‑blocking reads/writes.

Configuring Tomcat NIO

To enable NIO, set the connector's protocol attribute to org.apache.coyote.http11.Http11NioProtocol. The default maximum connections are 10,000, and the maxThreads attribute controls the size of the executor thread pool. In BIO mode, maxConnections is ignored and aligns with maxThreads because each request gets its own thread.

NioEndpoint Component Diagram

The NioEndpoint consists of five parts: LimitLatch (connection limiter, default 10,000), Acceptor (accepts new connections with a single thread), Poller (polls events, thread count = Math.min(2, Runtime.getRuntime().availableProcessors())), SocketProcessor (processes ready sockets), and Executor (thread pool defined by maxThreads).

NioEndpoint Execution Sequence Diagram

The sequence diagram (steps 1‑11) shows the lifecycle from binding the server socket to handling a request. Key steps include binding the IP/port, starting the acceptor thread, accepting connections, configuring channels as non‑blocking, registering them with the poller, creating PollerEvent objects, starting poller threads, selecting ready keys, and finally dispatching SocketProcessor to worker threads.

Step‑by‑Step Source Code Walkthrough

Initialize connection limits (default 10,000 for NIO).

Start the acceptor thread.

Accept new connections via ServerSocketChannel.accept().

Set the accepted channel to non‑blocking mode.

Construct a NioChannel object.

Register the channel with a poller thread.

Create a PollerEvent and add it to the event queue.

Start the poller thread.

Extract new PollerEvent objects and register them with the selector.

Invoke Selector.select() to wait for events.

When a SelectionKey is ready, create a SocketProcessor and submit it to the worker thread pool.

NioBlockingSelector and BlockPoller

The NioSelectorPool maintains a BlockPoller thread that uses an auxiliary selector for write‑ready events. When socket.write() returns 0, the socket is registered for OP_WRITE on this auxiliary selector, and the BlockPoller repeatedly polls until the socket becomes writable, then notifies the worker thread to continue writing. This design reduces context switches and offloads work from the main selector.

Performance Observations

Benchmark results show that NIO mainly benefits scenarios with many long‑lived connections, as it allows a small number of threads to hold a large number of connections. When the payload per request is tiny, BIO and NIO exhibit similar latency because the bottleneck shifts to CPU or network rather than I/O. The article also notes occasional early‑stage errors in NIO tests due to poller thread saturation under sudden spikes.

Conclusion

NIO optimizes network I/O by maximizing CPU utilization and minimizing thread count. However, its advantage appears only when the system’s bottleneck is I/O‑related; otherwise, the performance gain is negligible. Understanding the distinction between I/O and CPU bottlenecks is essential for choosing the appropriate model.

References

http://tomcat.apache.org/tomcat-7.0-doc/config/http.html

http://gearever.iteye.com/blog/1844203

《Tomcat内核设计剖析》

《深入理解计算机操作系统》

《UNIX网络编程》卷1