Build a Simple Go RPC Framework – A Beginner-Friendly Project

The author created a simple Go RPC framework named easyrpc (source on GitHub) to demonstrate core concepts such as graceful server shutdown, packet framing, network protocol design, connection reuse, serialization, compression, and basic Go data structures.

What you will learn

Graceful server termination

Packet framing (handling of sticky packets)

Designing a custom network protocol

Client‑side connection reuse for concurrent calls

Serialization and deserialization

Data compression and decompression

Code modularisation

Familiarity with Go networking APIs and basic data structures

RPC basics

Remote Procedure Call (RPC) enables a program on one machine to invoke a method on another machine as if it were local. The article outlines the typical flow: start a server on machine B, start a client on machine A, encode request parameters, send the packet, decode on the server, invoke the method, encode the result, and send it back.

Code structure

The project contains about 1,127 lines of Go code, with the core located in three directories: rpcclient, rpcmsg, and rpcserver. The rpcmsg package defines the RPCMsg struct, which holds the header, sequence number, object name, method name, and payload.

type RPCMsg struct {
    Header     [HEADER_LEN]byte // 5‑byte fixed header
    Seq        int64           // request ID
    ObjectName string
    MethodName string
    Payload    []byte
}

The header encodes five fields: magic number, protocol version, message type (0 = request, 1 = response), compression type (0 = none, 1 = zlib, 2 = snappy, 3 = lz4), and serialization type (0 = Gob, 1 = JSON).

Sending a message

func (t *RPCMsg) SendMsg(w io.Writer) error {
    // write 5‑byte header
    if _, err := w.Write(t.Header[:]); err != nil { return err }
    // write sequence ID (8 bytes)
    if err := binary.Write(w, binary.BigEndian, uint64(t.Seq)); err != nil { return err }
    // calculate and write total length of variable fields
    totalLen := DATA_LEN + uint32(len(t.ObjectName)) + DATA_LEN + uint32(len(t.MethodName)) + DATA_LEN + uint32(len(t.Payload))
    if err := binary.Write(w, binary.BigEndian, totalLen); err != nil { return err }
    // write lengths and contents of ObjectName, MethodName, Payload sequentially
    // ... (omitted for brevity)
    return nil
}

Receiving a message

func (t *RPCMsg) RecvMsg(r io.Reader) error {
    // read fixed 5‑byte header
    if _, err := io.ReadFull(r, t.Header[:]); err != nil { return err }
    if !t.Header.CheckMagicNumber() { return fmt.Errorf("magic number error: %v", t.Header[0]) }
    // read sequence ID
    var seqBytes [8]byte
    if _, err := io.ReadFull(r, seqBytes[:]); err != nil { return err }
    t.Seq = int64(binary.BigEndian.Uint64(seqBytes[:]))
    // read total length and then the variable‑length fields
    // ... (omitted for brevity)
    return nil
}

Server workflow

The server registers objects, listens for new TCP connections, spawns a goroutine per connection, reads complete RPCMsg packets, looks up the corresponding object and method, invokes it, encodes the result, compresses it, and sends the response back. Graceful shutdown is achieved by setting a shutdown flag, closing the listener, and spinning until all active connections finish.

Client workflow

The client establishes a single TCP connection, generates stub functions with reflect.MakeFunc, and for each call it builds an RPCMsg, assigns a unique sequence ID, sends the packet, and stores a waitMsg in a local map keyed by the ID. A dedicated goroutine continuously reads responses, matches them to the waiting calls, and signals completion.

Concurrency and ordering

Because multiple goroutines may share the same connection, the client serialises writes with a mutex (implemented in rpcclient/client.go). Responses can arrive out of order, so the client cannot read a response immediately after sending; instead it caches the sequence IDs and delivers the correct result to the awaiting call.

Overall, the article provides a step‑by‑step walkthrough of building a functional RPC system in Go, illustrating protocol design, binary encoding, error handling, graceful shutdown, and concurrent request processing.