Inside Netty's Write and Flush Mechanism: How Java Handles Network I/O

Overview

The article explains Netty's core write and flush logic, focusing on how outbound events travel through the pipeline, how data is buffered, and how the reactor thread interacts with the socket.

Write Event Propagation

When ctx.write(msg) is called, ChannelHandlerContext.write locates the next ChannelOutboundHandler using findContextOutbound(MASK_WRITE), wraps the message, and invokes invokeWrite. If the handler is not yet added, the event skips to the previous handler.

private void invokeWrite(Object msg, ChannelPromise promise) {
    try {
        ((ChannelOutboundHandler) handler()).write(this, msg, promise);
    } catch (Throwable t) {
        notifyOutboundHandlerException(t, promise);
    }
}

ChannelOutboundBuffer

Netty stores pending data in ChannelOutboundBuffer, a singly‑linked list of Entry objects. Each Entry holds the message, its size, a ChannelPromise, and bookkeeping fields such as pendingSize. The buffer tracks total pending bytes ( totalPendingSize) and enforces high/low watermarks (default 64 KB / 32 KB) to control writability.

public final void addMessage(Object msg, int size, ChannelPromise promise) {
    Entry e = Entry.newInstance(msg, size, total(msg), promise);
    if (tailEntry == null) {
        flushedEntry = null;
    } else {
        tailEntry.next = e;
    }
    tailEntry = e;
    if (unflushedEntry == null) {
        unflushedEntry = e;
    }
    incrementPendingOutboundBytes(e.pendingSize, false);
}

Flush Process

Calling ctx.flush() moves unflushedEntry to flushedEntry, then ChannelOutboundBuffer.addFlush() marks the range of entries to be sent. The reactor’s HeadContext finally invokes unsafe.flush(), which calls ChannelOutboundBuffer.addFlush() and then flush0() to write data to the socket.

protected final void flush() {
    assertEventLoop();
    ChannelOutboundBuffer out = outboundBuffer;
    if (out == null) return;
    out.addFlush();
    flush0();
}

Write Loop Details

During doWrite, Netty converts pending ByteBuf objects to JDK ByteBuffer arrays, respects maxBytesPerGatheringWrite (initially SO_SNDBUF*2), and attempts up to WRITE_SPIN_COUNT (default 16) writes. After each loop Netty adjusts the gathering size based on actual bytes written.

int writeSpinCount = config().getWriteSpinCount();
do {
    if (in.isEmpty()) { clearOpWrite(); return; }
    ByteBuffer[] buffers = in.nioBuffers(1024, maxBytesPerGatheringWrite);
    int nioBufferCnt = in.nioBufferCount();
    switch (nioBufferCnt) {
        case 0: writeSpinCount -= doWrite0(in); break;
        case 1: ...; break;
        default: ...; break;
    }
} while (writeSpinCount > 0);
incompleteWrite(writeSpinCount < 0);

OP_WRITE Handling

If a write returns 0 (socket send buffer full), Netty registers OP_WRITE on the selector via setOpWrite(). When the selector later signals writability, the reactor calls forceFlush(), which simply re‑invokes flush0(). After all data is sent, clearOpWrite() removes the interest to avoid endless notifications.

writeAndFlush Shortcut

The method ctx.writeAndFlush(msg) internally calls write(msg, true, promise), which first propagates a write event and then immediately a flush event using invokeWriteAndFlush. This combines the two steps into a single pipeline traversal.

public ChannelFuture writeAndFlush(Object msg, ChannelPromise promise) {
    write(msg, true, promise);
    return promise;
}

Overall, the article demystifies Netty’s asynchronous I/O pipeline, showing how it balances throughput, memory usage, and reactor fairness while providing hooks for custom handlers.