Unlock Go 1.26’s New Goroutine Scheduling Metrics for Better Observability

Background

The Go runtime/metrics package has provided memory and GC statistics since Go 1.16, but until now it lacked any metrics for the Goroutine scheduler, making it hard to observe scheduling behavior in production.

Pain Points

When a service slows down, typical Grafana dashboards show normal CPU, memory, and GC metrics, yet the root cause may be Goroutine scheduling problems that are invisible without dedicated metrics.

How many Goroutines have been created?

How many are queued for execution?

Are any stuck in system calls?

Is the thread count exploding?

New Proposal

The need for scheduler observability was raised in 2016 (issue #15490) and finally implemented in Go 1.26. The proposal mirrors the existing MemStats approach, exposing counters for Goroutine creation, current count, runnable, running, waiting, and thread usage.

Total Goroutines created since program start: /sched/goroutines-created:goroutines Goroutines in system calls or cgo (approx.): /sched/goroutines/not-in-go:goroutines Runnable but not yet executing (approx.): /sched/goroutines/runnable:goroutines Currently running (approx.): /sched/goroutines/running:goroutines Waiting for a resource such as I/O (approx.): /sched/goroutines/waiting:goroutines Active OS threads owned by the runtime: /sched/threads/total:threads All counters are uint64 and represent approximate values because exact tracking would require locking and degrade performance. The sum of state‑specific counts may not equal the total Goroutine count due to sampling timing.

Usage Example

The following program demonstrates reading the new metrics with runtime/metrics and printing them.

package main

import (
    "fmt"
    "runtime/metrics"
    "time"
)

func main() {
    // launch some goroutines to simulate workload
    go work()
    // let them run for a short while
    time.Sleep(100 * time.Millisecond)

    fmt.Println("Goroutine scheduling metrics:")
    printMetric("/sched/goroutines-created:goroutines", "Total created")
    printMetric("/sched/goroutines:goroutines", "Current alive")
    printMetric("/sched/goroutines/not-in-go:goroutines", "Syscall/CGO")
    printMetric("/sched/goroutines/runnable:goroutines", "Runnable")
    printMetric("/sched/goroutines/running:goroutines", "Running")
    printMetric("/sched/goroutines/waiting:goroutines", "Waiting for resource")

    fmt.Println("
Thread metrics:")
    printMetric("/sched/gomaxprocs:threads", "Max P")
    printMetric("/sched/threads/total:threads", "Current threads")
}

func printMetric(name string, descr string) {
    sample := []metrics.Sample{{Name: name}}
    metrics.Read(sample)
    // simplified error handling for demo purposes
    fmt.Printf("  %s: %v
", descr, sample[0].Value.Uint64())
}

func work() {
    // placeholder for real work (network, computation, etc.)
}

Running the program yields output similar to:

Goroutine scheduling metrics:
  Total created: 52
  Current alive: 12
  Syscall/CGO: 0
  Runnable: 0
  Running: 4
  Waiting for resource: 8

Thread metrics:
  Max P: 8
  Current threads: 4

The numbers show that 52 Goroutines were created, 12 remain alive, 4 are actively running, and 8 are waiting for I/O or other resources. No Goroutine is stuck in a system call, and the thread count is modest.

Conclusion

The proposal, first raised in 2016 and championed by core contributor Michael Knyszek, landed in Go 1.26 after a decade of community demand. Although the API is simple, the metrics dramatically improve observability of Goroutine scheduling in production environments, helping engineers pinpoint performance bottlenecks without guesswork.