Why Observability Is the Key to Simplifying Modern Microservices

Overview

Observability was created to handle the complexity of micro‑services architectures. It provides measurable insight into a system’s state and is a dimension of service governance alongside functionality, testability, and operability.

Observability consists of three primary measurement dimensions—Metric, Logging, Tracing—and a fourth, Profiling.

Metric : aggregated statistics such as QPS, latency, error rate.

Logging : discrete records of system actions, classified by severity levels (DEBUG, INFO, WARN, ERROR, FATAL).

Tracing : call‑chain data (spans) that shows request flow through components, with each span linked by Trace ID.

Profiling : continuous profiling (e.g., flame graphs) that reveals internal program state like stack calls and execution time, typically encoded as protocol‑buffer bytes.

The typical fault‑handling workflow follows the order: Metric → Tracing → Logging → Profiling.

Data Model

In the tracing world, OpenTracing and OpenCensus merged into OpenTelemetry, which now standardizes data models for Metric, Logging, and Tracing.

OpenTelemetry defines a unified protocol; it does not provide storage or visualization back‑ends. Exporters bridge OpenTelemetry data to systems such as Prometheus, Cortex, Loki, and Grafana Tempo.

Metric: stored in a distributed Prometheus‑compatible system (Cortex). Model =

LabelSet + Timestamp + Number

.

Logging: stored in Loki. Model =

LabelSet + Timestamp + String

(nanosecond precision).

Tracing: stored in Grafana Tempo, which accepts OpenTelemetry, Zipkin, Jaeger protocols. Model =

Operation Name + Start/End Timestamp + Attributes + Events + Parent + SpanContext

.

Profiling: stored in Loki (or similar) with model =

LabelSet + Timestamp + []byte

(protocol‑buffer payload).

Example Metric data (Counter, Gauge, Histogram, Summary) and a sample Loki LogQL query are shown below.

<code>{container="query-frontend",namespace="loki-dev"} |= "metrics.go" | logfmt | duration > 10s and throughput_mb < 500</code>

Example Tracing data is illustrated with diagrams (images omitted for brevity).

Fusion Scenarios

Metric & Tracing Fusion : Using Exemplar, a Trace ID can be added as a label to a metric, allowing Prometheus to retrieve tracing information.

<code>$ curl -g 'http://localhost:9090/api/v1/query_exemplar?query=test_exemplar_metric_total&start=2020-09-14T15:22:25.479Z&end=2020-09-14T15:23:25.479Z'</code>

Logging & Tracing Fusion : SDKs that emit tracing IDs embed those IDs in log entries, enabling correlation between logs and traces.

Metric & Profiling Fusion : Profiling IDs can be stored as Exemplars, making them queryable via Prometheus and visualizable in Grafana with a pprof panel.

These integrations illustrate how observability signals can be combined to provide richer insight into modern cloud‑native applications.

Source: https://mirror.xyz/0xFd007bb46C47D8600C139E34Df9DfceC86F0B319/hw60dfH7YMtM3jd5dT22spTpPGSS7T8yxskkddTXXro