Design and Implementation of Jetty ManagedSelector and ExecutionStrategy

This article, authored by the Vivo Internet Server Team, explains the design and implementation of ManagedSelector and ExecutionStrategy in Jetty. By comparing Jetty’s approach with the native select call, it reveals Jetty’s thread‑optimization philosophy.

Jetty, like Tomcat, is a Java web container composed of a set of Connectors, Handlers, and a ThreadPool. Its Connectors use the NIO model, where the core component is a Selector . Jetty wraps the native Java Selector with its own ManagedSelector .

1. Traditional Selector Implementation

Typical NIO programming separates I/O detection and request handling into two threads: one thread continuously calls select() to detect ready channels, and another thread processes the events after they are wrapped in a Runnable . This producer‑consumer model avoids interference between detection and processing but suffers from cache inefficiency and thread‑switch overhead because the detection thread may run on a different CPU core than the processing thread.

2. Jetty’s ManagedSelector

Jetty combines production and consumption of I/O events in the same thread, allowing the same CPU core to handle both detection and processing, thus better utilizing CPU caches and reducing context switches. The class definition of ManagedSelector (excerpt) is shown below:

public class ManagedSelector extends ContainerLifeCycle implements Dumpable {
    // Atomic flag indicating whether the selector has been started
    private final AtomicBoolean _started = new AtomicBoolean(false);
    // Indicates whether the selector is currently blocked on select()
    private boolean _selecting = false;
    // Reference to the manager that controls the lifecycle of several ManagedSelectors
    private final SelectorManager _selectorManager;
    // Identifier of this ManagedSelector
    private final int _id;
    // Core execution strategy that decides whether producer and consumer run in the same thread
    private final ExecutionStrategy _strategy;
    // The underlying Java Selector
    private Selector _selector;
    // Queue of selector‑update tasks
    private Deque
_updates = new ArrayDeque<>();
    private Deque
_updateable = new ArrayDeque<>();
    ...
}

The SelectorUpdate interface represents a task that updates the selector’s state (e.g., registering a channel or changing interest ops):

/**
 * A selector update to be done when the selector has been woken.
 */
public interface SelectorUpdate {
    void update(Selector selector);
}

Updates are submitted to the selector via a task object, for example:

_selector.submit(_updateKeyAction);

The Selectable interface defines callbacks used by ManagedSelector when an I/O event becomes ready:

public interface Selectable {
    // Returns a Runnable that will process the ready I/O event
    Runnable onSelected();
    // Called after all events have been processed
    void updateKey();
}

3. ExecutionStrategy and Thread Optimization

Jetty’s ExecutionStrategy abstracts how tasks produced by the selector are executed. It contains an inner Producer interface whose produce() method returns a Runnable . The produced task may be run by the current thread or handed off to a thread pool.

public interface ExecutionStrategy {
    void dispatch(); // used only in HTTP/2, ignored here
    void produce();
    interface Producer {
        Runnable produce();
    }
}

Jetty implements several concrete strategies:

ProduceConsume (PC) : The same thread both produces and consumes tasks. Simple but can become a bottleneck.

ProduceExecuteConsume (PEC) : Production runs in one thread, consumption in another. Improves parallelism but incurs cache misses and thread‑switch cost.

ExecuteProduceConsume (EPC) : Production runs in a thread pool, consumption may run in the same thread. Utilizes CPU cache but can suffer from thread starvation if tasks block.

EatWhatYouKill (EWYK) : Jetty’s adaptive strategy. When the thread pool is abundant, it behaves like EPC; under heavy load it falls back to PEC to avoid starvation.

The SelectorProducer inner class of ManagedSelector demonstrates how the selector loops, processes ready events, handles pending SelectorUpdate tasks, and finally calls select() again:

private class SelectorProducer implements ExecutionStrategy.Producer {
    private Set
_keys = Collections.emptySet();
    private Iterator
_cursor = Collections.emptyIterator();
    @Override
    public Runnable produce() {
        while (true) {
            // If there are ready I/O events, obtain the Runnable via Selectable and return it
            Runnable task = processSelected();
            if (task != null) return task;
            // Otherwise, handle pending selector‑update tasks
            processUpdates();
            updateKeys();
            // Continue to select for new I/O events
            if (!select()) return null;
        }
    }
}

4. Summary

The article shows that Jetty‑9’s ManagedSelector and ExecutionStrategy enable a flexible, cache‑friendly I/O processing model. By preferring the EPC (or EWYK) strategy, Jetty keeps production and consumption on the same thread whenever possible, maximizing CPU cache reuse and minimizing context‑switch overhead. The presented concepts provide useful ideas for performance tuning in high‑concurrency I/O‑bound applications.

References:

Class EatWhatYouKill

Eat What You Kill

Thread Starvation with Eat What You Kill