SOFARPC Performance Optimization (Part 1): Custom Protocol, Netty Tuning, Serialization, and Proxy Enhancements

The speaker, Lei Zhiyuan from Ant Financial Middleware, introduces the first part of the SOFARPC performance optimization series, aiming to share practical improvements and provide code demos for verification.

SOFARPC’s architecture is layered from the transport layer (Http2Transport, BoltTransport) up through protocol, serialization, filter, router, and cluster components.

Custom Communication Protocol : The BOLT protocol is presented as a custom solution that allows richer header metadata, separate header/body serialization, CRC checks, versioning, multiple serialization formats, and security features, addressing limitations of Http2, Dubbo, and Rest.

Netty Performance Parameter Optimization includes enabling SO_REUSEPORT/SO_REUSEADDR for port reuse, TCP_FASTOPEN to exchange data during the three‑way handshake, disabling Nagle’s algorithm with TCP_NODELAY, turning on SO_KEEPALIVE, configuring WRITE_BUFFER_WATER_MARK to control back‑pressure, adjusting workerGroup thread count, setting EventLoop ioRatio (recommended 70), and tuning SO_BACKLOG (e.g., 1024) to balance connection queues.

Connection Keep‑Alive and Management : Beyond kernel keep‑alive, SOFARPC uses Netty idle events for single‑direction heartbeats (client‑to‑server) and supports multiple connections per address to maximize throughput while avoiding excessive connections.

Serialization Selection : For homogeneous Java environments, Hessian offers a good balance of compatibility and performance; for cross‑language scenarios, Protobuf is recommended. Compatibility considerations for interface changes are emphasized.

IO Thread‑Pool Batch Decoding : Netty’s ByteToMessageDecoder can batch‑decode messages, reducing pipeline invocations. Example implementation:

if (batchSwitch) {    ArrayList<Object> ret = new ArrayList<Object>(size);    for (int i = 0; i < size; i++) {        ret.add(out.get(i));    }    ctx.fireChannelRead(ret);} else {    for (int i = 0; i < size; i++) {        ctx.fireChannelRead(out.get(i));    }}

Client Proxy Performance Optimization : Benchmark results show Javassist bytecode proxy achieving the best latency (≈7.9 ns/op) compared to JDK dynamic proxy, CGLIB, ByteBuddy, and others. The recommendation is to prefer Javassist bytecode for RPC client proxies.

In summary, building a high‑performance RPC framework like SOFARPC involves careful protocol design, Netty tuning, appropriate serialization, efficient decoding, and optimized proxy generation, all validated with JMH benchmarks and real‑world testing.