Designing Complex Protocols: String & JSON Implementations with Large Message Handling

Overview

The chapter explores the design and implementation of complex network protocols. It first revisits the StringProtocol introduced earlier, then presents a JSON‑based protocol using FastJSON, and finally discusses strategies for processing large message bodies.

6.1 String Protocol Implementation Details

6.1.1 Protocol Format Design

StringProtocol uses a length‑field prefix to delimit messages. The wire format is:

+----------+------------------+
| Length   | String Content   |
| (4 bytes)| (variable bytes) |
+----------+------------------+

Advantages:

Clear message boundaries : the length field precisely indicates where a message ends.

Support for variable‑length messages : any string size can be transmitted.

Efficient parsing : only two read operations are needed to obtain the length and the payload.

6.1.2 Core Implementation Analysis

The core of StringProtocol is shown below (generics escaped):

public class StringProtocol implements Protocol<String> {
    private final Charset charset;
    public StringProtocol(Charset charset) { this.charset = charset; }
    public StringProtocol() { this(StandardCharsets.UTF_8); }
    @Override
    public String decode(ByteBuffer readBuffer, AioSession session) {
        FixedLengthFrameDecoder decoder = session.getAttachment();
        if (decoder != null) {
            if (!decoder.decode(readBuffer)) return null; // decoding not finished
            ByteBuffer byteBuffer = decoder.getBuffer();
            session.setAttachment(null);
            return convert(byteBuffer, byteBuffer.capacity());
        }
        if (readBuffer.remaining() < Integer.BYTES) return null;
        readBuffer.mark();
        int length = readBuffer.getInt();
        if (length + Integer.BYTES > readBuffer.capacity()) {
            FixedLengthFrameDecoder fixedLengthFrameDecoder = new FixedLengthFrameDecoder(length);
            session.setAttachment(fixedLengthFrameDecoder);
            return null;
        }
        if (length > readBuffer.remaining()) {
            readBuffer.reset();
            return null;
        }
        return convert(readBuffer, length);
    }
    private String convert(ByteBuffer byteBuffer, int length) {
        byte[] b = new byte[length];
        byteBuffer.get(b);
        return new String(b, charset);
    }
}

Key points:

Decoder management : each session stores a FixedLengthFrameDecoder as an attachment to handle fragmented large messages.

Resource cleanup : after a message is fully decoded, the attachment is cleared.

Boundary checks : the code validates that enough bytes are available for the length field and the payload, handling half‑packet scenarios.

6.2 JSON Protocol Design and FastJSON Integration

6.2.1 JSON Protocol Format

The JSON protocol also adopts a length‑field prefix:

+----------+------------------+
| Length   | JSON Content     |
| (4 bytes)| (variable bytes) |
+----------+------------------+

6.2.2 JSON Protocol Implementation

public class JsonProtocol<T> implements Protocol<T> {
    private final Class<T> clazz;
    public JsonProtocol(Class<T> clazz) { this.clazz = clazz; }
    @Override
    public T decode(ByteBuffer readBuffer, AioSession session) {
        if (readBuffer.remaining() < Integer.BYTES) return null;
        readBuffer.mark();
        int length = readBuffer.getInt();
        if (length > readBuffer.remaining()) {
            readBuffer.reset();
            return null;
        }
        byte[] data = new byte[length];
        readBuffer.get(data);
        try {
            return JSON.parseObject(data, clazz);
        } catch (Exception e) {
            throw new DecoderException("JSON parsing failed", e);
        }
    }
}

A usage example demonstrates a server that sends and receives Message objects encoded as JSON:

public class Message {
    private String from;
    private String to;
    private String content;
    // getters and setters omitted
}
AioQuickServer<Message> server = new AioQuickServer<>(
    8888,
    new JsonProtocol<>(Message.class),
    new MessageProcessor<Message>() {
        @Override
        public void process(AioSession session, Message msg) {
            System.out.println("Received from " + msg.getFrom() + " message: " + msg.getContent());
        }
    }
);
// Client side
Message message = new Message();
message.setFrom("client");
message.setTo("server");
message.setContent("Hello, smart-socket!");
String json = JSON.toJSONString(message);
byte[] data = json.getBytes(StandardCharsets.UTF_8);
session.writeBuffer().writeInt(data.length);
session.writeBuffer().write(data);
session.writeBuffer().flush();

6.3 Large Message Body Handling Strategy

When transmitting large files or data blocks, the chapter recommends using the session's attachment to store decoding state. The LargeMessageProtocol illustrates this approach:

public class LargeMessageProtocol implements Protocol<String> {
    @Override
    public String decode(ByteBuffer readBuffer, AioSession session) {
        LargeMessageDecodeState state = session.getAttachment();
        if (state == null) {
            if (readBuffer.remaining() < Integer.BYTES) return null;
            int length = readBuffer.getInt();
            if (length <= 0 || length > 10 * 1024 * 1024) {
                throw new DecoderException("Invalid message length: " + length);
            }
            state = new LargeMessageDecodeState(length);
            session.setAttachment(state);
            if (length > readBuffer.remaining()) return null;
        }
        int remaining = Math.min(readBuffer.remaining(), state.getRemainingLength());
        byte[] data = new byte[remaining];
        readBuffer.get(data);
        state.appendData(data);
        if (state.isComplete()) {
            session.setAttachment(null);
            return new String(state.getData(), StandardCharsets.UTF_8);
        }
        return null;
    }
    private static class LargeMessageDecodeState {
        private final int totalLength;
        private final ByteArrayOutputStream buffer;
        public LargeMessageDecodeState(int totalLength) {
            this.totalLength = totalLength;
            this.buffer = new ByteArrayOutputStream(totalLength);
        }
        public boolean isComplete() { return buffer.size() >= totalLength; }
        public int getRemainingLength() { return totalLength - buffer.size(); }
        public void appendData(byte[] data) { buffer.write(data, 0, data.length); }
        public byte[] getData() { return buffer.toByteArray(); }
    }
}

The implementation validates length (max 10 MB), creates a state object for incremental accumulation, and clears the attachment once the full message is received.

6.4 Protocol Design Best Practices

6.4.1 Error Handling

A robust protocol should catch parsing errors and provide clear diagnostics. Example:

public class RobustProtocol implements Protocol<String> {
    @Override
    public String decode(ByteBuffer readBuffer, AioSession session) {
        try {
            if (readBuffer.remaining() < Integer.BYTES) return null;
            int length = readBuffer.getInt();
            if (length < 0 || length > 1024 * 1024) {
                throw new DecoderException("Message length exceeds limit: " + length);
            }
            if (length > readBuffer.remaining()) return null;
            byte[] data = new byte[length];
            readBuffer.get(data);
            return new String(data, StandardCharsets.UTF_8);
        } catch (Exception e) {
            System.err.println("Protocol parsing error: " + e.getMessage());
            throw new DecoderException("Protocol parsing failed", e);
        }
    }
}

6.4.2 Performance Optimizations

Object reuse : avoid frequent allocation of temporary objects.

Buffer management : use direct buffers and minimize copying.

Asynchronous processing : offload time‑consuming tasks to separate threads or thread pools.

6.5 Chapter Summary

The chapter covered:

StringProtocol : detailed analysis of its length‑prefixed format, core code, decoder management, and boundary checks.

JSON protocol : design of a JSON‑based message format, FastJSON integration, and a complete client‑server example.

Large message handling : session‑attachment technique and a stateful decoder for messages up to 10 MB.

Best practices : error handling strategies and performance tips such as object reuse, efficient buffer usage, and asynchronous processing.

After studying this material, readers should be able to understand complex protocol design concepts, implement JSON protocols, correctly handle large payloads, and write robust, high‑performance protocol implementations.