Mastering High‑Performance Structured Logging in Go with Uber’s Zap

Overview

Zap is an open‑source, high‑performance structured logging library for Go, originally built for Uber’s internal systems and released in 2016. It is widely adopted in performance‑critical Go projects such as Kubernetes, Istio, and InfluxData.

Key Features

High performance – several orders of magnitude faster than the standard log package, even under high concurrency.

Structured logging – arbitrary fields can be attached to each log entry, facilitating later analysis.

Dynamic level control – log levels can be changed at runtime to filter output.

Multiple encoders – built‑in JSON and console encoders, with hook support for custom formats.

Automatic log rotation – split logs by size, date, or other criteria.

Installation

Add Zap to your module with the standard Go command: go get -u go.uber.org/zap If the dependency already appears in go.mod, it will look like: go.uber.org/zap v1.27.0 // indirect Additional optional dependencies used in the examples:

github.com/natefinch/lumberjack v2.0.0+incompatible // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect

Basic Usage

A minimal test demonstrates creating a production logger, writing an info, warning, and error entry, and flushing the buffer:

func TestLogZap(t *testing.T) {
    logger, _ := zap.NewProduction() // create a new Logger instance
    defer logger.Sync()              // ensure buffered entries are flushed
    logger.Info("FunTester,例子", zap.String("name", "FunTester"), zap.Int("score", 100))
    logger.Info("warn FunTester coming!!!")
    logger.Warn("warn FunTester coming!!!")
    logger.Error("error FunTester coming!!!")
}

Console output (JSON format) includes the log level, timestamp, caller, message, and optional fields such as stacktrace for error level:

{"level":"info","ts":1717310460.23924,"caller":"test/zap_test.go:16","msg":"This is an info message","category":"example","counter":1}
{... "level":"warn", ...}
{... "level":"error", "stacktrace":"funtester/test.TestLogZap
\t/..."}

Default Level Configuration

Zap’s built‑in level behavior is:

Debug – caller info, no stacktrace
Info  – caller info, no stacktrace
Warn  – caller info, no stacktrace
Error – caller info, includes stacktrace
DPanic, Panic, Fatal – caller info, includes stacktrace

Sugared Logger

The Sugared Logger offers a more ergonomic, printf‑style API at a slight performance cost. Example:

func TestLogZapSugar(t *testing.T) {
    logger, _ := zap.NewProduction()
    defer logger.Sync()
    sugar := logger.Sugar()
    sugar.Infow("调用失败", "方法", "FunTester", "调用次数", 3, "时间单位", time.Second)
    sugar.Infof("调用方法失败 %s", "FunTester")
}

Output shows structured fields when using Infow and simple formatted text with Infof:

2024-06-02T14:57:28.298+0800 INFO test/zap_test.go:62 This is a custom logger info message {"category":"custom","counter":1}
2024-06-02T14:57:28.299+0800 WARN  test/zap_test.go:66 This is a custom logger warning message
2024-06-02T14:57:28.299+0800 ERROR test/zap_test.go:67 This is a custom logger error message
2024-06-02T14:57:28.299+0800 INFO  test/zap_test.go:68 This is a structured log message {"key1":"value1","key2":42}

Fine‑Grained Level Control

By customizing zapcore.EncoderConfig and an atomic level, you can filter which levels are emitted to the console:

encoderConfig := zapcore.EncoderConfig{
    TimeKey:    "T",
    LevelKey:   "L",
    NameKey:    "log",
    CallerKey:  "C",
    MessageKey: "msg",
    StacktraceKey: "stacktrace",
    EncodeLevel: zapcore.CapitalLevelEncoder,
    EncodeTime:  zapcore.ISO8601TimeEncoder,
    EncodeDuration: zapcore.StringDurationEncoder,
    EncodeCaller: zapcore.ShortCallerEncoder,
}
encoder := zapcore.NewConsoleEncoder(encoderConfig)
atomicLevel := zap.NewAtomicLevel()
atomicLevel.SetLevel(zap.InfoLevel) // only Info and above are printed
core := zapcore.NewCore(encoder, zapcore.Lock(os.Stdout), atomicLevel)
logger := zap.New(core, zap.AddCaller(), zap.Development())
logger.Warn("打印警告日志")
logger.Error("打印错误日志")
logger.Info("打印结构化日志", zap.String("key1", "FunTester"), zap.Int("key2", 22))

Running the test prints only the warning, error, and info entries to the console.

Logging to Files

To persist logs, create a file, configure a JSON encoder, and combine it with a console encoder using zapcore.NewTee:

logDir := "logs"
os.MkdirAll(logDir, 0755)
logFile := filepath.Join(logDir, "app.log")
file, err := os.OpenFile(logFile, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil { panic(err) }
encoderConfig := zapcore.EncoderConfig{ /* same as above */ }
jsonEncoder := zapcore.NewJSONEncoder(encoderConfig)
consoleEncoder := zapcore.NewConsoleEncoder(encoderConfig)
writeSyncer := zapcore.AddSync(file)
consoleSyncer := zapcore.AddSync(os.Stdout)
atomicLevel := zap.NewAtomicLevel()
atomicLevel.SetLevel(zap.InfoLevel)
core := zapcore.NewCore(jsonEncoder, writeSyncer, atomicLevel)
consoleCore := zapcore.NewCore(consoleEncoder, consoleSyncer, atomicLevel)
combinedCore := zapcore.NewTee(core, consoleCore)
logger := zap.New(combinedCore, zap.AddCaller(), zap.Development())
defer logger.Sync()
logger.Warn("打印警告日志")
logger.Error("打印错误日志")
logger.Info("打印结构化日志", zap.String("key1", "FunTester"), zap.Int("key2", 22))

The console shows the same formatted messages, while app.log contains JSON entries with keys L, T, C, msg, and any custom fields.

Log Rotation

Using the lumberjack package, Zap can automatically rotate log files by size, age, or number of backups:

writeSyncer := zapcore.AddSync(&lumberjack.Logger{
    Filename:   "logs/app.log",
    MaxSize:    10,   // MB per file
    MaxBackups: 5,    // keep last 5 files
    MaxAge:     30,   // days
    Compress:   true, // gzip old files
})
encoderConfig := zapcore.EncoderConfig{ /* same as above */ }
encoder := zapcore.NewJSONEncoder(encoderConfig)
atomicLevel := zap.NewAtomicLevel()
atomicLevel.SetLevel(zap.InfoLevel)
core := zapcore.NewCore(encoder, writeSyncer, atomicLevel)
logger := zap.New(core, zap.AddCaller(), zap.Development())
logger.Warn("打印警告日志")
logger.Error("打印错误日志")
logger.Info("打印结构化日志", zap.String("key1", "FunTester"), zap.Int("key2", 22))

The logger writes JSON entries to logs/app.log, automatically creating new files when the size exceeds 10 MB and compressing old files.