Investigation of Sporadic Netty Message Send Delays Exceeding One Second

Problem description: The business uses Netty to send messages, and during performance testing some messages occasionally take more than 1 second to send. CPU usage was low, indicating the bottleneck was not resource‑related.

Relevant performance‑tracking code:

int sendBytes = ((ByteBuf)msg).readableBytes();

long beginTime = System.currentTimeMillis();

ChannelFuture writeFuture = ctx.writeAndFlush(msg);

writeFuture.addListener((f) ->

{

long costTime = System.currentTimeMillis() - beginTime;

if (costTime > 1000)

logger.warn("Send msg cost time is : " + costTime);

totalSendBytes.getAndAdd(sendBytes);

});

Analysis of possible causes:

Stop‑the‑World (STW) GC pauses, such as Full GC, which can be diagnosed with -XX:+PrintGCApplicationStoppedTime , -XX:+PrintSafepointStatistics , and related flags.

Unexpected blocking in a business ChannelHandler outbound operation, which can stall the Netty NioEventLoop thread.

Slow or stalled peer reads, causing TCP send buffers to fill; netstat can be used to inspect Recv‑Q and Send‑Q values.

Root‑cause discovery: The custom ChannelHandler performed a conditional check that, when not satisfied, invoked Thread.sleep(1000) in a loop until the condition became true. This sleep blocked the NioEventLoop thread, resulting in random >1 s send delays.

Conclusion and recommendations:

Avoid any blocking operations (sleep, wait, synchronous I/O, database access, disk I/O) inside Netty ChannelHandler s.

Replace while‑loop + sleep patterns with event‑driven or asynchronous callbacks to prevent thread blockage.