Understanding Go Memory Leaks and Using pprof for Profiling

Memory leaks are common in languages like C/C++ where manual allocation and deallocation are required, but Go uses a garbage collector (GC) that automatically manages memory, making traditional leaks rare. However, Go applications can still show continuously increasing memory usage due to unreasonable object usage, such as a global map that grows without ever being cleared.

The primary way to investigate these issues is to use Go's built‑in pprof profiling tool. First, import the net/http and net/http/pprof packages and start an HTTP server on a chosen port:

import "net/http"
import _ "net/http/pprof"

func main() {
    // ...
    go func() {
        http.ListenAndServe("0.0.0.0:8080", nil)
    }()
    // ...
}

After the program runs, you can fetch profiling data via HTTP, for example:

go tool pprof http://127.0.0.1:8080/debug/pprof/allocs

Running top inside the pprof interactive console shows allocation stacks and their memory consumption.

pprof works by sampling allocations at a configurable rate. By default, Go samples every 512 KiB of allocated memory (the MemProfileRate variable). This sampling interval explains why pprof reports less memory than the actual physical usage.

You can adjust the sampling frequency using the GODEBUG=memprofilerate=1 environment variable, which forces sampling on every allocation, at the cost of performance.

Go's memory allocator is based on tcmalloc, which eliminates external fragmentation but still has internal fragmentation (about 12.5%). The runtime also keeps additional memory for idle spans, stack usage, and internal structures such as mcache, mspan, and GC metadata.

Memory allocated via cgo is not accounted for by Go's profiler because it uses the C library's malloc/free, which the Go runtime cannot track.

A real‑world example shows a process reporting ~15 GB of RSS in top , while pprof only sees about 6 GB of Go allocations. Examining runtime.ReadMemStats reveals large idle heap spans, stack memory, and runtime metadata contributing to the discrepancy.

Summary :

Go memory problems are usually caused by unreasonable business logic rather than classic leaks.

pprof can resolve the majority of these issues.

Sampling under‑reports total allocations but still highlights problematic code paths.

Adjusting memprofilerate can increase sampling granularity, though it is rarely needed.

tcmalloc’s internal fragmentation is around 12 %.

Idle heap spans, stack usage, and runtime structures can consume significant memory.

cgo‑allocated memory is invisible to Go profilers and should be considered only when cgo is used.