Master Reading Data from io.Reader in Go: Simple Techniques & Code Samples

Overview

Go is popular for high‑concurrency cloud development. This article explains how to read data from an io.Reader in Go, covering basic Read semantics and several convenient utilities.

Direct Read with Read(p []byte)

The io.Reader interface defines a single method:

<code>type Reader interface {
    Read(p []byte) (n int, err error)
}
</code>

Each call reads up to len(p) bytes.

The method returns the number of bytes read and either nil , an error, or io.EOF when the stream ends.

Read never changes the size of p .

io.EOF signals that no more data is available.

Example that reads from a TCP connection in a loop:

<code>package main

import (
    "fmt"
    "io"
    "net"
)

func main() {
    conn, err := net.Dial("tcp", "www.findme.wang:80")
    if err != nil {
        fmt.Println("dial error:", err)
        return
    }
    defer conn.Close()

    httpReq := `GET / HTTP/1.0
Host: www.findme.wang
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36
Content-Type:application/x-www-form-urlencoded
Content-Length:0
`
    _, err = fmt.Fprintf(conn, httpReq)
    if err != nil {
        fmt.Println("http request error:", err)
        return
    }

    var rsData []byte
    for {
        tmp := make([]byte, 512)
        n, err := conn.Read(tmp)
        if n >= 0 {
            rsData = append(rsData, tmp[:n]...)
        }
        if err == io.EOF {
            fmt.Println("data read complete")
            break
        } else if err != nil {
            fmt.Println("read error:", err)
            break
        }
    }
    fmt.Println("read length:", len(rsData))
}
</code>

Using io.Copy

io.Copy copies all data from a Reader to a Writer . A simple writer can be implemented to collect the bytes.

<code>func Copy(dst Writer, src Reader) (written int64, err error) {
    return copyBuffer(dst, src, nil)
}
</code>

Custom writer example:

<code>type MyWriter struct {
    data []byte
}

func (m *MyWriter) Write(p []byte) (n int, err error) {
    if m.data == nil {
        m.data = make([]byte, 0)
    }
    if len(p) != 0 {
        m.data = append(m.data, p...)
    }
    return len(p), nil
}

func main() {
    conn, err := net.Dial("tcp", "www.findme.wang:80")
    if err != nil {
        fmt.Println("dial error:", err)
        return
    }
    defer conn.Close()
    // send request (omitted for brevity)
    w := new(MyWriter)
    n, err := io.Copy(w, conn)
    if err != nil {
        fmt.Println("read err", err)
    }
    fmt.Println("read length:", n)
}
</code>

Using strings.Builder

strings.Builder implements the Writer interface and efficiently builds a string.

<code>var sb strings.Builder
n, err := io.Copy(&sb, conn)
fmt.Println(sb.String())
fmt.Println("read length:", n)
</code>

Using ioutil.ReadAll

ioutil.ReadAll reads the entire content of a Reader in one call.

<code>data, err := ioutil.ReadAll(conn)
if err != nil {
    fmt.Println("read err", err)
}
fmt.Println(string(data))
fmt.Println("read length:", len(data))
</code>

Additional io utilities

The io package also provides functions such as io.ReadAtLeast and io.ReadFull for reading a minimum number of bytes.

<code>func ReadAtLeast(r Reader, buf []byte, min int) (n int, err error) {
    if len(buf) < min {
        return 0, ErrShortBuffer
    }
    for n < min && err == nil {
        var nn int
        nn, err = r.Read(buf[n:])
        n += nn
    }
    if n >= min {
        err = nil
    } else if n > 0 && err == EOF {
        err = ErrUnexpectedEOF
    }
    return
}

func ReadFull(r Reader, buf []byte) (n int, err error) {
    return ReadAtLeast(r, buf, len(buf))
}
</code>