Master Go’s io Package: Readers, Writers, and Stream I/O Explained

Introduction

In Go, input and output are performed using primitives that treat data as readable or writable byte streams. The io package provides the io.Reader and io.Writer interfaces for these operations.

io.Reader

io.Reader

represents a reader that reads data from a source into a buffer. Any type that implements the single method Read(p []byte) (n int, err error) satisfies the interface.

type Reader interface {
    Read(p []byte) (n int, err error)
}

The Read method returns the number of bytes read and an error; when the source is exhausted it returns io.EOF.

Using a Reader

The following example creates a string reader with strings.NewReader and reads the data in 4‑byte chunks.

func main() {
    reader := strings.NewReader("Clear is better than clever")
    p := make([]byte, 4)
    for {
        n, err := reader.Read(p)
        if err != nil {
            if err == io.EOF {
                fmt.Println("EOF:", n)
                break
            }
            fmt.Println(err)
            os.Exit(1)
        }
        fmt.Println(n, string(p[:n]))
    }
}

Output shows the final read may return fewer bytes than the buffer size.

Implementing a Custom Reader

The custom alphaReader filters out non‑alphabetic characters while reading.

type alphaReader struct {
    src string // source string
    cur int    // current read position
}

func newAlphaReader(src string) *alphaReader { return &alphaReader{src: src} }

func alpha(r byte) byte {
    if (r >= 'A' && r <= 'Z') || (r >= 'a' && r <= 'z') {
        return r
    }
    return 0
}

func (a *alphaReader) Read(p []byte) (int, error) {
    if a.cur >= len(a.src) {
        return 0, io.EOF
    }
    x := len(a.src) - a.cur
    bound := len(p)
    if x < bound {
        bound = x
    }
    buf := make([]byte, bound)
    n := 0
    for n < bound {
        if ch := alpha(a.src[a.cur]); ch != 0 {
            buf[n] = ch
        }
        n++
        a.cur++
    }
    copy(p, buf)
    return n, nil
}

func main() {
    reader := newAlphaReader("Hello! It's 9am, where is the sun?")
    p := make([]byte, 4)
    for {
        n, err := reader.Read(p)
        if err == io.EOF {
            break
        }
        fmt.Print(string(p[:n]))
    }
    fmt.Println()
}

Output:

HelloItsamwhereisthesun

Composing Multiple Readers

A reader can wrap another reader to reuse or hide lower‑level implementation details. The example below adapts alphaReader to accept any io.Reader as its source.

type alphaReader struct {
    reader io.Reader
}

func newAlphaReader(reader io.Reader) *alphaReader { return &alphaReader{reader: reader} }

func alpha(r byte) byte { /* same filter as before */ }

func (a *alphaReader) Read(p []byte) (int, error) {
    n, err := a.reader.Read(p)
    if err != nil {
        return n, err
    }
    buf := make([]byte, n)
    for i := 0; i < n; i++ {
        if ch := alpha(p[i]); ch != 0 {
            buf[i] = ch
        }
    }
    copy(p, buf)
    return n, nil
}

func main() {
    reader := newAlphaReader(strings.NewReader("Hello! It's 9am, where is the sun?"))
    p := make([]byte, 4)
    for {
        n, err := reader.Read(p)
        if err == io.EOF {
            break
        }
        fmt.Print(string(p[:n]))
    }
    fmt.Println()
}

This design lets alphaReader work with any io.Reader, such as os.File.

io.Writer

io.Writer

represents a writer that writes data from a buffer to a destination. The interface requires a single method Write(p []byte) (n int, err error).

type Writer interface {
    Write(p []byte) (n int, err error)
}

Using a Writer

The example writes several strings into a bytes.Buffer, which implements io.Writer.

func main() {
    proverbs := []string{"Channels orchestrate mutexes serialize", "Cgo is not Go", "Errors are values", "Don't panic"}
    var writer bytes.Buffer
    for _, p := range proverbs {
        n, err := writer.Write([]byte(p))
        if err != nil || n != len(p) {
            fmt.Println("failed to write data")
            os.Exit(1)
        }
    }
    fmt.Println(writer.String())
}

Implementing a Custom Writer

The chanWriter writes bytes into a channel, implementing both io.Writer and io.Closer.

type chanWriter struct {
    ch chan byte
}

func newChanWriter() *chanWriter { return &chanWriter{make(chan byte, 1024)} }

func (w *chanWriter) Chan() <-chan byte { return w.ch }

func (w *chanWriter) Write(p []byte) (int, error) {
    for _, b := range p {
        w.ch <- b
    }
    return len(p), nil
}

func (w *chanWriter) Close() error { close(w.ch); return nil }

func main() {
    writer := newChanWriter()
    go func() {
        defer writer.Close()
        writer.Write([]byte("Stream "))
        writer.Write([]byte("me!"))
    }()
    for c := range writer.Chan() {
        fmt.Printf("%c", c)
    }
    fmt.Println()
}

Other Useful Types and Functions in io

os.File

implements both io.Reader and io.Writer, allowing file I/O with any io‑based code. Example of writing to a file:

func main() {
    proverbs := []string{"Channels orchestrate mutexes serialize
", "Cgo is not Go
", "Errors are values
", "Don't panic
"}
    file, err := os.Create("./proverbs.txt")
    if err != nil { fmt.Println(err); os.Exit(1) }
    defer file.Close()
    for _, p := range proverbs {
        n, err := file.Write([]byte(p))
        if err != nil || n != len(p) { fmt.Println("failed to write data"); os.Exit(1) }
    }
    fmt.Println("file write done")
}

Reading the same file:

func main() {
    file, err := os.Open("./proverbs.txt")
    if err != nil { fmt.Println(err); os.Exit(1) }
    defer file.Close()
    p := make([]byte, 4)
    for {
        n, err := file.Read(p)
        if err == io.EOF { break }
        fmt.Print(string(p[:n]))
    }
}

The standard streams os.Stdout, os.Stdin, and os.Stderr are also *os.File values and thus satisfy io.Writer or io.Reader as appropriate.

io.Copy

io.Copy(dst, src)

copies data from a Reader to a Writer, handling the read‑loop and io.EOF automatically.

func main() {
    // copy buffer to file
    file, _ := os.Create("./proverbs.txt")
    defer file.Close()
    io.Copy(file, proverbs)
    // copy file to stdout
    f, _ := os.Open("./proverbs.txt")
    defer f.Close()
    io.Copy(os.Stdout, f)
}

io.WriteString

Conveniently writes a string to any io.Writer.

func main() {
    file, _ := os.Create("./magic_msg.txt")
    defer file.Close()
    io.WriteString(file, "Go is fun!")
}

Pipes

io.PipeReader

and io.PipeWriter simulate an in‑memory pipe. Data written to the writer can be read from the reader, typically in separate goroutines.

func main() {
    proverbs := bytes.NewBufferString("Channels orchestrate mutexes serialize
Cgo is not Go
")
    piper, pipew := io.Pipe()
    go func() {
        defer pipew.Close()
        io.Copy(pipew, proverbs)
    }()
    io.Copy(os.Stdout, piper)
    piper.Close()
}

Buffered I/O

The bufio package provides buffered readers and writers. Example of line‑by‑line reading:

func main() {
    file, _ := os.Open("./planets.txt")
    defer file.Close()
    reader := bufio.NewReader(file)
    for {
        line, err := reader.ReadString('
')
        if err != nil {
            if err == io.EOF { break }
            fmt.Println(err); os.Exit(1)
        }
        fmt.Print(line)
    }
}

ioutil (now io and os )

Utility functions such as ioutil.ReadFile read an entire file into a byte slice.

func main() {
    data, err := ioutil.ReadFile("./planets.txt")
    if err != nil { fmt.Println(err); os.Exit(1) }
    fmt.Printf("%s", data)
}

Conclusion

The article demonstrated how to use io.Reader and io.Writer interfaces for stream I/O in Go, how to implement custom readers and writers, and introduced related utilities such as io.Copy, pipes, buffered I/O, and file operations. Although it is a brief overview, it provides a solid foundation for working with Go’s streaming I/O.