Uncovering Netty’s TCP Connection Closure: From Normal Shutdown to Half‑Close Bug Fix

Overview

The article provides a detailed analysis of how Netty processes TCP connection termination, covering normal closure, abnormal closure (RST), and graceful half‑close, and reveals a bug that prevents the active side from reading data after a half‑close when SO_LINGER is enabled.

1. Normal TCP Connection Close

Netty follows the TCP four‑handshake: the client calls ctx.channel().close(), the kernel sends a FIN, the server replies with ACK and inserts EOF, and both sides transition through FIN_WAIT1, FIN_WAIT2, CLOSE_WAIT, LAST_ACK, and TIME_WAIT. Netty handles the critical steps in AbstractNioByteChannel#read where allocHandle.lastBytesRead() becomes -1, triggering closeOnRead(pipeline).

public final void read() {
    // ... allocate buffer ...
    int bytes = doReadBytes(buf);
    if (bytes <= 0) {
        buf.release();
        close = bytes < 0;
        if (close) readPending = false;
        break;
    }
    // ... fireChannelRead ...
}

When close is true, Netty executes the close logic, clears ChannelOutboundBuffer, fires ChannelInactive and ChannelUnregistered events, and finally closes the underlying SocketChannel.

2. Abnormal TCP Close (RST)

If the peer sends an RST packet, Netty receives an IOException during doReadBytes. The exception is caught in handleReadException, which fires ExceptionCaught and then calls closeOnRead to clean up the channel similarly to a normal close.

private void handleReadException(ChannelPipeline p, ByteBuf buf, Throwable cause, boolean close, RecvByteBufAllocator.Handle alloc) {
    if (buf != null) {
        if (buf.isReadable()) {
            readPending = false;
            p.fireChannelRead(buf);
        } else {
            buf.release();
        }
    }
    alloc.readComplete();
    p.fireChannelReadComplete();
    p.fireExceptionCaught(cause);
    if (close || cause instanceof IOException) {
        closeOnRead(p);
    }
}

3. TCP Half‑Close (Graceful Shutdown)

Half‑close uses the shutdownOutput() system call (equivalent to SHUT_WR) to close only the write direction. Netty’s implementation mirrors the normal close flow but does not close the read side, allowing the peer to continue sending data.

public ChannelFuture shutdownOutput() {
    final EventLoop loop = eventLoop();
    if (loop.inEventLoop()) {
        ((AbstractUnsafe) unsafe()).shutdownOutput(promise);
    } else {
        loop.execute(() -> ((AbstractUnsafe) unsafe()).shutdownOutput(promise));
    }
    return promise;
}

After shutdownOutput, Netty clears the ChannelOutboundBuffer, invokes doShutdownOutput() (which calls javaChannel().shutdownOutput()), and fires a user event ChannelOutputShutdownEvent so handlers can react.

private void shutdownOutput(ChannelPromise promise, Throwable cause) {
    if (!promise.setUncancellable()) return;
    if (outboundBuffer == null) { promise.setFailure(new ClosedChannelException()); return; }
    this.outboundBuffer = null; // prevent further writes
    // ... handle SO_LINGER if needed ...
    doShutdownOutput();
    promise.setSuccess();
    pipeline.fireUserEventTriggered(ChannelOutputShutdownEvent.INSTANCE);
}

The peer receives a FIN, enters CLOSE_WAIT, and can still read data. Netty then fires ChannelInputShutdownEvent on the server side, allowing it to send remaining data before finally closing.

4. The SO_LINGER Interaction

When SO_LINGER is set with a positive timeout, Netty’s prepareToClose() deregisters the channel from the selector and uses GlobalEventExecutor to avoid blocking the reactor thread during the delayed close. However, this logic is appropriate for close() but not for shutdownOutput(), which never blocks.

5. The Bug

With SO_LINGER enabled, calling shutdownOutput() incorrectly invokes prepareToClose(), which deregisters the channel. Consequently, the half‑closed side stops receiving OP_READ events and cannot read data sent by the peer during FIN_WAIT2. This defeats the purpose of a graceful half‑close.

6. Fixing the Bug

The fix removes the call to prepareToClose() from the shutdownOutput path, ensuring the channel remains registered and can continue to receive reads. The patch also adds proper unit tests to verify that data can be read after a half‑close when SO_LINGER is set.

// Updated shutdownOutput implementation
private void shutdownOutput(ChannelPromise promise, Throwable cause) {
    if (!promise.setUncancellable()) return;
    if (outboundBuffer == null) { promise.setFailure(new ClosedChannelException()); return; }
    this.outboundBuffer = null;
    // No prepareToClose() here – half‑close must stay registered
    doShutdownOutput();
    promise.setSuccess();
    pipeline.fireUserEventTriggered(ChannelOutputShutdownEvent.INSTANCE);
}

The change was submitted as PR #11982 , passed Netty’s CI checks, and was merged in version 4.1.73.Final.

7. Full Half‑Close Flow

1. Active side calls shutdownOutput() → FIN sent. 2. Passive side receives FIN, isInputShutdown0() is false, so it calls shutdownInput() and fires ChannelInputShutdownEvent. 3. Passive side may write remaining data, then receives ChannelInputShutdownReadComplete and closes. 4. Active side receives FIN, reads -1, triggers closeOnRead, fires ChannelInactive and finally closes.

Conclusion

The article explains Netty’s complete TCP termination logic, the role of SO_LINGER, the half‑close mechanism, and how a subtle bug prevented proper half‑close reads. The provided fix restores graceful shutdown capabilities and demonstrates the importance of aligning low‑level socket semantics with Netty’s event‑driven architecture.