Master Go’s io Package: Readers, Writers, Copy & Pipe Explained

Introduction

In Go, input and output are handled through the io package, which provides a set of interfaces and utilities for reading and writing data. The package groups related functionality into several sub‑packages such as io, io/ioutil, fmt, bufio, and net.

Reader and Writer Interfaces

The two most fundamental interfaces.

Reader Interface

type Reader interface {
    Read(p []byte) (n int, err error)
}

The Read method fills the supplied buffer p and returns the number of bytes read n and any error err. Successful reads return n == len(p) and err == nil. Errors include EOF and other read failures.

Writer Interface

type Writer interface {
    Write(p []byte) (n int, err error)
}

The Write method writes the contents of p to the underlying data stream, returning the number of bytes written and any error.

Common Implementations

os.File implements both Reader and Writer strings.Reader implements Reader bufio.Reader / bufio.Writer implement Reader and Writer respectively

bytes.Buffer implements both interfaces

compress/gzip.Reader and Writer

crypto/cipher.StreamReader / StreamWriter

crypto/tls.Conn implements both

encoding/csv.Reader and Writer

mime/multipart.Part implements Reader net.Conn implements both Reader and

Writer

ReaderAt and WriterAt Interfaces

ReaderAt

type ReaderAt interface {
    ReadAt(p []byte, off int64) (n int, err error)
}

ReadAt

reads from a specific offset off without changing the underlying read position. It must either fill the buffer completely or return err == EOF; partial reads without EOF are not allowed.

WriterAt

type WriterAt interface {
    WriteAt(p []byte, off int64) (n int, err error)
}

WriteAt

writes len(p) bytes at the given offset. If fewer bytes are written, a non‑nil error must be returned.

ReaderFrom and WriterTo Interfaces

ReaderFrom

type ReaderFrom interface {
    ReadFrom(r Reader) (n int64, err error)
}

Copies data from r until EOF, returning the total bytes copied.

WriterTo

type WriterTo interface {
    WriteTo(w Writer) (n int64, err error)
}

Writes the entire content to w, returning the number of bytes written.

Copy and CopyBuffer Functions

Copy

func Copy(dst Writer, src Reader) (written int64, err error) {
    return copyBuffer(dst, src, nil)
}

Copy

reads from src and writes to dst using an internal 32 KB buffer, returning the total bytes copied. It treats EOF as a successful termination, not an error.

CopyBuffer

func CopyBuffer(dst Writer, src Reader, buf []byte) (written int64, err error) {
    if buf != nil && len(buf) == 0 {
        panic("empty buffer in io.CopyBuffer")
    }
    return copyBuffer(dst, src, buf)
}

CopyBuffer

allows the caller to provide a custom buffer, which can be pooled via sync.Pool to reduce allocations.

Pipe Function

io.Pipe()

creates a synchronous in‑memory pipe connecting a PipeReader and a PipeWriter. Writes block until the corresponding read consumes the data, enabling streaming without intermediate buffering.

func Pipe() (*PipeReader, *PipeWriter) {
    p := &pipe{wrCh: make(chan []byte), rdCh: make(chan int), done: make(chan struct{})}
    return &PipeReader{p}, &PipeWriter{p}
}

The implementation uses channels to synchronize reads and writes, ensuring that each write is matched by a read.

Practical Examples

Examples demonstrate using io.Copy to download large files efficiently, and using io.Pipe together with a JSON encoder to stream a large payload without allocating a full buffer.