Loki for Cloud‑Native: Architecture Deep Dive and Step‑by‑Step Binary Deployment

In the cloud‑native era, expanding microservice clusters generate explosive log volumes, and traditional ELK stacks consume high resources and are complex to deploy; Loki is introduced as a lightweight log aggregation system that indexes only labels, minimizing resource usage while meeting collection, storage, and query needs in Kubernetes environments.

Loki’s core advantages include low‑resource operation without external dependencies, seamless Kubernetes integration, compressed storage that reduces cost, a simple PromQL‑like query language (LogQL), and strong horizontal scalability.

The architecture follows a micro‑service design that can run as a single binary or be distributed across multiple nodes. It consists of four logical layers—collection, transport, storage, and query—implemented by components such as Promtail (collector), the Loki server (Distributor, Ingester, Query Frontend, Querier, Storage, Compactor), and Grafana for visualization.

Component details:

Promtail runs on log‑producing nodes, reads files or container logs, adds custom labels (e.g., service, node, environment), and forwards batches to the Distributor.

Distributor hashes label values to route logs to the appropriate Ingester, ensuring logs with the same labels land in the same shard.

Ingester compresses logs, stores them in memory, and periodically flushes them to the storage layer (Index + Chunks) while also serving real‑time queries.

Query Frontend parses and optimizes incoming queries, dispatches them to Ingester for recent logs and to Querier for historic logs, then aggregates the results.

Querier reads compressed log chunks from storage, applies filters and aggregations, and returns the final data.

Storage separates metadata (Index) from raw log data (Chunks). Index can use TSDB (preferred after Loki 2.0) or BoltDB; Chunks are time‑sharded and can reside on local disks or object stores such as S3, GCS, or Azure Blob, which underpins Loki’s low‑cost storage claim.

Compactor periodically merges and recompresses chunks and removes expired data according to retention policies, improving storage efficiency and query performance.

Data flow (five steps):

Promtail collects logs and adds labels (e.g., service=user-service, node=node-1).

Promtail sends the labeled logs to the Distributor.

Distributor hashes the labels and forwards logs to the corresponding Ingester.

Ingester compresses logs, stores them in memory, and flushes them to the storage layer (Index + Chunks).

Users query via Grafana; Query Frontend parses the request, routes it to Ingester (real‑time) and Querier (historical), aggregates results, and displays them in Grafana.

Binary deployment guide (Docker Compose):

mkdir evaluate-loki
cd evaluate-loki

wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/loki-config.yaml -O loki-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/alloy-local-config.yaml -O alloy-local-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/docker-compose.yaml -O docker-compose.yaml

docker compose up -d
# Sample output showing containers started
✔ Network evaluate-loki_loki          Created   0.1s
✔ Container evaluate-loki-minio-1    Started   0.6s
✔ Container evaluate-loki-flog-1      Started   0.6s
✔ Container evaluate-loki-backend-1   Started   0.8s
✔ Container evaluate-loki-write-1     Started   0.8s
✔ Container evaluate-loki-read-1      Started   0.8s
✔ Container evaluate-loki-gateway-1   Started   1.1s
✔ Container evaluate-loki-grafana-1   Started   1.4s
✔ Container evaluate-loki-alloy-1      Started   1.4s

Access Grafana at http://<em>IP</em>:3000 with the default credentials admin/admin. Ensure the host ports do not conflict with existing services, then select the desired container, log type, and label filters in the Grafana UI.