Building a Custom Redis in Go: Implementing the RESP Network Protocol

The EasyRedis series aims to build a functional Redis service in Go, and this second part focuses on the network request serialization protocol (RESP). The goal is to dissect and implement the protocol rather than merely describe it.

Redis Protocol Format

RESP defines five data types, each identified by a leading character:

Simple String ( +) – e.g., +OK\r\n Error ( -) – e.g., -ERR Invalid Syntax\r\n Integer ( :) – e.g., :1\r\n Bulk String ( $) – binary‑safe, e.g., $3\r\nSET\r\n (with $-1 representing nil)

Array ( *) – e.g., *3\r\n$3\r\nSET\r\n$3\r\nkey\r\n$5\r\nvalue\r\n When a command like set key value is sent via redis-cli, the client actually transmits the RESP array shown above.

Code Implementation

The previous article introduced a redisHandler to manage connections. The core handling uses the protocol parsing rules defined here.

func (t *TCPServer) handleConn(conn net.Conn) {
    // ...
    logger.Debugf("accept new conn %s", conn.RemoteAddr().String())
    // TODO: handle connection
    t.redisHandler.Handle(context.Background(), conn)
}

The handler reads data via parser.ParseStream(conn), which splits input on \r\n and feeds parsed payloads into a channel, following a producer‑consumer model.

func (h *RedisHandler) Handle(ctx context.Context, conn net.Conn) {
    h.activeConn.Store(conn, struct{}{})
    outChan := parser.ParseStream(conn)
    for payload := range outChan {
        if payload.Err != nil {
            if payload.Err == io.EOF || payload.Err == io.ErrUnexpectedEOF || strings.Contains(payload.Err.Error(), "use of closed network connection") {
                h.activeConn.Delete(conn)
                conn.Close()
                logger.Warn("client closed:" + conn.RemoteAddr().String())
                return
            }
            errReply := protocal.NewGenericErrReply(payload.Err.Error())
            _, err := conn.Write(errReply.ToBytes())
            if err != nil {
                h.activeConn.Delete(conn)
                conn.Close()
                logger.Warn("client closed:" + conn.RemoteAddr().String() + " err info: " + err.Error())
                return
            }
            continue
        }
        if payload.Reply == nil {
            logger.Error("empty payload")
            continue
        }
        reply, ok := payload.Reply.(*protocal.MultiBulkReply)
        if !ok {
            logger.Error("require multi bulk protocol")
            continue
        }
        logger.Debugf("%q", string(reply.ToBytes()))
        result := h.engine.Exec(conn, reply.RedisCommand)
        if result != nil {
            conn.Write(result.ToBytes())
        } else {
            conn.Write(protocal.NewUnknownErrReply().ToBytes())
        }
    }
}

The parsing logic resides in redis/parser.go, particularly the parse function:

func parse(r io.Reader, out chan<- *Payload) {
    defer func() {
        if err := recover(); err != nil {
            logger.Error(err, string(debug.Stack()))
        }
    }()
    reader := bufio.NewReader(r)
    for {
        line, err := reader.ReadBytes('
')
        if err != nil {
            out &-amp; Payload{Err: err}
            close(out)
            return
        }
        length := len(line)
        if length <= 2 || line[length-2] != '\r' {
            continue // ignore blank lines
        }
        line = bytes.TrimSuffix(line, []byte{'\r', '
'})
        switch line[0] {
        case '*':
            err := parseArrays(line, reader, out)
            if err != nil {
                out &-amp; Payload{Err: err}
                close(out)
                return
            }
        default:
            args := bytes.Split(line, []byte{' '})
            out &-amp; Payload{Reply: protocal.NewMultiBulkReply(args)}
        }
    }
}

Because Bulk Strings are binary‑safe, the parser reads the exact byte length using io.ReadFull instead of relying on \r\n as a delimiter:

// Read the specified length plus the trailing 

body := make([]byte, dataLen+2)
_, err := io.ReadFull(reader, body)
if err != nil {
    return err
}
lines = append(lines, body[:len(body)-2]) // strip 

Demo

Using the official redis-cli to connect to EasyRedis, the following commands are correctly parsed:

*2
$3
get
$9
easyredis

*3
$3
set
$9
easyredis
$1
1

The next article will cover in‑memory KV storage for the parsed commands.

Extended Knowledge

The previous article used bufio.NewScanner for config parsing; this part uses bufio.NewReader to read raw bytes from the network.

In Go, NewReader creates a buffered Reader for low‑level byte reads, while NewScanner creates a Scanner for convenient text tokenization.

func NewReader(rd io.Reader) *Reader

NewReader

provides a 4 KB default buffer and is suitable for binary data such as RESP payloads.

func NewScanner(r io.Reader) *Scanner

NewScanner

builds on NewReader and is geared toward line‑oriented text processing.

Choosing between them depends on whether you need raw byte access ( NewReader) or higher‑level text scanning ( NewScanner).