Understanding Loki: Architecture, Benefits, and Comparison with ELK

When applications or nodes running in a container cloud encounter problems, the usual approach is to rely on Prometheus metrics and alerts, but this alone does not provide enough information to diagnose issues.

Metrics indicate that a value has crossed a threshold, triggering an alert, yet operators still need to examine logs to understand the root cause, such as a pod whose memory usage spikes.

Traditional ELK stacks require users to switch between Kibana for logs and Grafana for metrics, which hurts user experience. Loki was created to minimize the cost of switching between metrics and logs, thereby reducing incident response time.

Full‑text indexing solutions like ELK consume significant resources and are often overkill for simple time‑range or label‑based queries. Loki’s second goal is to strike a balance between query simplicity and expressive power.

Full‑text search incurs high indexing and storage costs; alternative designs such as OKlog offer cheaper, simpler operations but lack convenient querying. Loki’s third goal is to provide a cost‑effective solution.

The overall architecture of Loki is intentionally simple: it reuses Prometheus‑style labels as indexes, allowing the same label selectors to query both logs and metrics, which reduces storage for log indexes.

Promtail runs as a DaemonSet on each Kubernetes node, discovers logs via the Kubernetes API, attaches the appropriate metadata, and forwards them to Loki.

Write path: the Distributor receives logs from Promtail, then forwards them to Ingester components, which batch and compress logs into Gzip‑compressed chunks. Chunks are replicated (default three times) and flushed to storage when they reach size or time thresholds.

The Ingester builds chunks, appends compressed log data, and creates new empty chunks after flushing.

Reading is handled by the Querier, which receives a time range and label selector, consults the index to find matching chunks, and performs distributed greps, also pulling the latest unflushed data from Ingester.

Scalability: index storage can be backed by Cassandra, Bigtable, or DynamoDB, while chunks reside in object storage. Distributor and Querier are stateless; Ingester is stateful but rebalances chunks when nodes are added or removed, ensuring resilience.