Mastering Sticky Packet and Splitting Issues in Netty RPC Applications

In RPC frameworks that maintain long TCP connections (e.g., Dubbo), sticky packet and split packet problems inevitably arise because multiple requests share a single connection. When the amount of data sent is smaller than the OS buffer, TCP may combine several requests into one packet (sticky packet). Conversely, when a request exceeds the buffer size, TCP splits it into multiple packets (split packet).

1. Sticky Packet and Split Packet

The main cause is the TCP buffer size. Small messages are merged, forming a sticky packet; large messages are divided, forming a split packet. The following diagram illustrates three typical scenarios:

A and B each exactly fill the TCP buffer, so they are sent as two separate packets.

A and B are sent in quick succession with small sizes, causing them to be merged into a single packet.

B is large and gets split into B_1 and B_2; the small B_2 may be merged with A.

2. Common Solutions

Four typical approaches are used to solve these problems:

Fix the length of each packet (e.g., 1024 bytes) and pad with spaces if necessary.

Append a fixed delimiter (e.g., "\r\n") to each packet and split on that delimiter.

Include a length field in the packet header and read the full message only after the specified length is received.

Design a custom protocol that defines its own framing rules.

3. Netty Solutions

3.1 FixedLengthFrameDecoder

For fixed‑length framing, Netty provides FixedLengthFrameDecoder. It reads a predefined number of bytes; if the data is insufficient, it waits for the next chunk. The corresponding encoder ( FixedLengthFrameEncoder) pads short messages with spaces.

public class EchoServer {
    public void bind(int port) throws InterruptedException {
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        try {
            ServerBootstrap bootstrap = new ServerBootstrap();
            bootstrap.group(bossGroup, workerGroup)
                     .channel(NioServerSocketChannel.class)
                     .option(ChannelOption.SO_BACKLOG, 1024)
                     .handler(new LoggingHandler(LogLevel.INFO))
                     .childHandler(new ChannelInitializer<SocketChannel>() {
                         @Override
                         protected void initChannel(SocketChannel ch) throws Exception {
                             // Add FixedLengthFrameDecoder with length 20
                             ch.pipeline().addLast(new FixedLengthFrameDecoder(20));
                             // Convert bytes to String
                             ch.pipeline().addLast(new StringDecoder());
                             // Custom encoder to pad to length 20
                             ch.pipeline().addLast(new FixedLengthFrameEncoder(20));
                             // Business logic handler
                             ch.pipeline().addLast(new EchoServerHandler());
                         }
                     });
            ChannelFuture future = bootstrap.bind(port).sync();
            future.channel().closeFuture().sync();
        } finally {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    }
    public static void main(String[] args) throws InterruptedException {
        new EchoServer().bind(8080);
    }
}

The pipeline adds FixedLengthFrameDecoder and StringDecoder for inbound data, and FixedLengthFrameEncoder for outbound data.

public class FixedLengthFrameEncoder extends MessageToByteEncoder<String> {
    private int length;
    public FixedLengthFrameEncoder(int length) { this.length = length; }
    @Override
    protected void encode(ChannelHandlerContext ctx, String msg, ByteBuf out) throws Exception {
        if (msg.length() > length) {
            throw new UnsupportedOperationException("message length is too large, it's limited " + length);
        }
        if (msg.length() < length) {
            msg = addSpace(msg);
        }
        ctx.writeAndFlush(Unpooled.wrappedBuffer(msg.getBytes()));
    }
    private String addSpace(String msg) {
        StringBuilder builder = new StringBuilder(msg);
        for (int i = 0; i < length - msg.length(); i++) {
            builder.append(" ");
        }
        return builder.toString();
    }
}

The server handler simply prints the received message and replies:

public class EchoServerHandler extends SimpleChannelInboundHandler<String> {
    @Override
    protected void channelRead0(ChannelHandlerContext ctx, String msg) throws Exception {
        System.out.println("server receives message: " + msg.trim());
        ctx.writeAndFlush("hello client!");
    }
}

The client mirrors the server setup, adding the same decoder/encoder and a handler that sends a greeting on connection and prints the server response.

public class EchoClient {
    public void connect(String host, int port) throws InterruptedException {
        EventLoopGroup group = new NioEventLoopGroup();
        try {
            Bootstrap bootstrap = new Bootstrap();
            bootstrap.group(group)
                     .channel(NioSocketChannel.class)
                     .option(ChannelOption.TCP_NODELAY, true)
                     .handler(new ChannelInitializer<SocketChannel>() {
                         @Override
                         protected void initChannel(SocketChannel ch) throws Exception {
                             ch.pipeline().addLast(new FixedLengthFrameDecoder(20));
                             ch.pipeline().addLast(new StringDecoder());
                             ch.pipeline().addLast(new FixedLengthFrameEncoder(20));
                             ch.pipeline().addLast(new EchoClientHandler());
                         }
                     });
            ChannelFuture future = bootstrap.connect(host, port).sync();
            future.channel().closeFuture().sync();
        } finally {
            group.shutdownGracefully();
        }
    }
    public static void main(String[] args) throws InterruptedException {
        new EchoClient().connect("127.0.0.1", 8080);
    }
}

public class EchoClientHandler extends SimpleChannelInboundHandler<String> {
    @Override
    protected void channelRead0(ChannelHandlerContext ctx, String msg) throws Exception {
        System.out.println("client receives message: " + msg.trim());
    }
    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        ctx.writeAndFlush("hello server!");
    }
}

3.2 LineBasedFrameDecoder and DelimiterBasedFrameDecoder

Netty also offers LineBasedFrameDecoder (splits on "\n" or "\r\n") and DelimiterBasedFrameDecoder (splits on a user‑defined delimiter). The encoder side must append the same delimiter.

ch.pipeline().addLast(new DelimiterBasedFrameDecoder(1024, Unpooled.wrappedBuffer("_$".getBytes())));
ch.pipeline().addLast(new StringDecoder());
ch.pipeline().addLast(new DelimiterBasedFrameEncoder("_$"));

3.3 LengthFieldBasedFrameDecoder and LengthFieldPrepender

For protocols that embed a length field, Netty provides LengthFieldBasedFrameDecoder and LengthFieldPrepender. The decoder extracts the length field and reads the full message; the prepender adds the length field before encoding.

Typical constructor parameters: maxFrameLength: maximum packet size. lengthFieldOffset: offset of the length field. lengthFieldLength: size of the length field. lengthAdjustment: adjustment for header length. initialBytesToStrip: bytes to discard (e.g., header).

Example server using JSON serialization with length‑field framing:

public class EchoServer {
    public void bind(int port) throws InterruptedException {
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        try {
            ServerBootstrap bootstrap = new ServerBootstrap();
            bootstrap.group(bossGroup, workerGroup)
                     .channel(NioServerSocketChannel.class)
                     .childHandler(new ChannelInitializer<SocketChannel>() {
                         @Override
                         protected void initChannel(SocketChannel ch) throws Exception {
                             ch.pipeline().addLast(new LengthFieldBasedFrameDecoder(1024, 0, 2, 0, 2));
                             ch.pipeline().addLast(new LengthFieldPrepender(2));
                             ch.pipeline().addLast(new JsonDecoder());
                             ch.pipeline().addLast(new JsonEncoder());
                             ch.pipeline().addLast(new EchoServerHandler());
                         }
                     });
            ChannelFuture f = bootstrap.bind(port).sync();
            f.channel().closeFuture().sync();
        } finally {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    }
    public static void main(String[] args) throws InterruptedException {
        new EchoServer().bind(8080);
    }
}

The JsonDecoder converts inbound ByteBuf to a User object, while JsonEncoder serializes a User back to JSON.

public class JsonDecoder extends MessageToMessageDecoder<ByteBuf> {
    @Override
    protected void decode(ChannelHandlerContext ctx, ByteBuf buf, List<Object> out) throws Exception {
        byte[] bytes = new byte[buf.readableBytes()];
        buf.readBytes(bytes);
        User user = JSON.parseObject(new String(bytes, CharsetUtil.UTF_8), User.class);
        out.add(user);
    }
}

public class JsonEncoder extends MessageToByteEncoder<User> {
    @Override
    protected void encode(ChannelHandlerContext ctx, User user, ByteBuf out) throws Exception {
        String json = JSON.toJSONString(user);
        ctx.writeAndFlush(Unpooled.wrappedBuffer(json.getBytes()));
    }
}

The server handler simply echoes the received User object:

public class EchoServerHandler extends SimpleChannelInboundHandler<User> {
    @Override
    protected void channelRead0(ChannelHandlerContext ctx, User user) throws Exception {
        System.out.println("receive from client: " + user);
        ctx.write(user);
    }
}

Client code mirrors the server, adding the same length‑field decoder/encoder and JSON codec, then sending a User instance on connection and printing the server response.

3.4 Custom Framing Handlers

If built‑in decoders do not satisfy a custom protocol, developers can extend LengthFieldBasedFrameDecoder / LengthFieldPrepender or implement ByteToMessageDecoder and MessageToByteEncoder directly to define bespoke framing logic.

4. Summary

The article first describes the principles behind sticky and split packet problems, then reviews four common mitigation strategies, and finally demonstrates how Netty’s various framing utilities—FixedLengthFrameDecoder, LineBasedFrameDecoder, DelimiterBasedFrameDecoder, LengthFieldBasedFrameDecoder, and custom handlers—can be applied with complete Java examples.