How to Build End-to-End Distributed Tracing in Go Microservices with OpenTelemetry and Jaeger

Why OpenTelemetry + Jaeger

OpenTelemetry follows cloud‑native open standards, avoids vendor lock‑in, and can switch exporters (e.g., to Grafana Tempo) by only changing configuration. It integrates with existing observability components such as Prometheus for metrics and Loki for logs.

Step 1 – Tracing module and configuration

A tracing package is added under internal/shared/ to initialize the OpenTelemetry TracerProvider and configure an OTLP HTTP exporter that sends trace data to Jaeger.

package tracing

import (
    "context"
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
    "go.opentelemetry.io/otel/propagation"
    "go.opentelemetry.io/otel/sdk/resource"
    "go.opentelemetry.io/otel/sdk/trace"
    semconv "go.opentelemetry.io/otel/semconv/v1.21.0"
)

// InitTracerProvider initializes and registers OpenTelemetry Tracer Provider.
func InitTracerProvider(serviceName, endpoint string) (func(context.Context) error, error) {
    exporter, err := otlptracehttp.New(context.Background(), otlptracehttp.WithEndpoint(endpoint), otlptracehttp.WithInsecure())
    if err != nil { return nil, err }
    res, err := resource.New(context.Background(), resource.WithAttributes(semconv.ServiceNameKey.String(serviceName)))
    if err != nil { return nil, err }
    tp := trace.NewTracerProvider(trace.WithBatcher(exporter), trace.WithResource(res))
    otel.SetTracerProvider(tp)
    otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(propagation.TraceContext{}, propagation.Baggage{}))
    return tp.Shutdown, nil
}

Step 2 – Configuration model

A TracingConfig struct is added to the global AppConfig to enable/disable tracing and set the collector endpoint.

type AppConfig struct {
    // ... other fields
    Tracing TracingConfig `yaml:"tracing,omitempty"`
}

type TracingConfig struct {
    Enable   bool   `yaml:"enable"`
    Endpoint string `yaml:"endpoint" // e.g., http://jaeger:14268/api/traces`
}

Step 3 – Service entry initialization

Each service's main.go loads the configuration and, if tracing is enabled, calls tracing.InitTracerProvider. The returned shutdown function is deferred to ensure graceful termination.

func main() {
    // load config
    appConfig := config.GetAppConfig()
    if appConfig.Tracing.Enable {
        shutdown, err := tracing.InitTracerProvider(serverName, appConfig.Tracing.Endpoint)
        if err != nil {
            logger.Error(err, "Failed to initialize tracer provider", serverName)
        }
        defer shutdown(context.Background())
    }
    // start service
}

Step 4 – Context propagation

HTTP (Gin)

import "go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin"

func main() {
    g := gin.Default()
    if appConfig.Tracing.Enable {
        g.Use(otelgin.Middleware(serverName))
    }
    // routes …
}

gRPC

import "go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"

// Server side
if appConfig.Tracing.Enable {
    s = grpc.NewServer(grpc.StatsHandler(otelgrpc.NewServerHandler()))
} else {
    s = grpc.NewServer()
}

// Client side
if appConfig.Tracing.Enable {
    opts = []grpc.DialOption{grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithStatsHandler(otelgrpc.NewClientHandler())}
} else {
    opts = []grpc.DialOption{grpc.WithTransportCredentials(insecure.NewCredentials())}
}

RabbitMQ

A custom carrier amqpHeadersCarrier implements propagation.TextMapCarrier to inject and extract trace context from AMQP headers.

type amqpHeadersCarrier map[string]interface{}

func (c amqpHeadersCarrier) Get(key string) string { /* … */ }
func (c amqpHeadersCarrier) Set(key, value string) { c[key] = value }
func (c amqpHeadersCarrier) Keys() []string { /* … */ }

// Publishing side
otel.GetTextMapPropagator().Inject(ctx, amqpHeadersCarrier(event.Headers))

// Consuming side
ctx := otel.GetTextMapPropagator().Extract(context.Background(), amqpHeadersCarrier(d.Headers))

Step 5 – Database tracing (GORM)

The OpenTelemetry GORM plugin is imported and registered when creating the gorm.DB instance.

import "gorm.io/plugin/opentelemetry/tracing"

func createGormDB(dialector gorm.Dialector, dbType string) (*gorm.DB, error) {
    db, err := gorm.Open(dialector, config)
    if err != nil { return nil, err }
    if err := db.Use(tracing.NewPlugin()); err != nil { return nil, err }
    return db, nil
}

Step 6 – Local validation – Deploy Jaeger

The docker-compose.yaml adds a Jaeger all‑in‑one service and configures each microservice to point to the OTLP collector.

services:
  jaeger:
    image: jaegertracing/all-in-one:latest
    container_name: jaeger
    ports:
      - "16686:16686"   # UI
      - "14268:14268"   # OTLP HTTP
    environment:
      - COLLECTOR_OTLP_ENABLED=true
  auth-svc:
    environment:
      - EASYMS_TRACING_ENABLE=true
      - EASYMS_TRACING_ENDPOINT=http://jaeger:14268/api/traces
    depends_on:
      - jaeger

Result

After rebuilding and launching the stack, opening http://localhost:16686 shows a complete trace graph for any request, revealing service flow, operation latency, and error propagation.

Future work

Smart sampling strategies to balance overhead and coverage.

Using OpenTelemetry Baggage to propagate business context such as user or tenant IDs.

Linking logs with trace IDs for seamless navigation between tracing and logging.

Source code

Git repositories:

https://github.com/louis-xie-programmer/easyms.golang

https://gitee.com/louis_xie/easyms.golang