Introduction to Loki: Architecture, Deployment, and Usage

When designing a container‑cloud logging solution, the author chose Grafana Loki over the heavier ELK/EFK stacks because Loki avoids full‑text indexing and stores only compressed logs with metadata tags, making it more cost‑effective and easier to operate.

Overview

Loki is a horizontally scalable, highly available, multi‑tenant log aggregation system that indexes only log stream labels, similar to Prometheus, and integrates natively with Grafana.

Architecture

Four Loki roles: querier, ingester, query‑frontend, distributor (selected via the -target flag).

Promtail acts as the log collector, analogous to Filebeat.

Log streams are identified by label sets; when labels change, new streams are created.

Read Path

Querier receives HTTP requests, forwards queries to ingesters, merges results, removes duplicates, and returns the final dataset.

Write Path

Distributor receives HTTP push requests, hashes streams, forwards data to appropriate ingesters, which write chunks to storage.

Deployment (Local Mode)

wget https://github.com/grafana/loki/releases/download/v2.2.1/loki-linux-amd64.zip
wget https://github.com/grafana/loki/releases/download/v2.2.1/promtail-linux-amd64.zip

Install Promtail:

$ mkdir -p /opt/app/{promtail,loki}
cat <
/opt/app/promtail/promtail.yaml
server:
  http_listen_port: 9080
  grpc_listen_port: 0
positions:
  filename: /var/log/positions.yaml
client:
  url: http://localhost:3100/loki/api/v1/push
scrape_configs:
- job_name: system
  static_configs:
  - targets: ["localhost"]
    labels:
      job: varlogs
      host: yourhost
      __path__: /var/log/*log
EOF
unzip promtail-linux-amd64.zip
mv promtail-linux-amd64 /opt/app/promtail/promtail
cat <
/etc/systemd/system/promtail.service
[Unit]
Description=promtail server
Wants=network-online.target
After=network-online.target
[Service]
ExecStart=/opt/app/promtail/promtail -config.file=/opt/app/promtail/promtail.yaml
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=promtail
[Install]
WantedBy=default.target
EOF
systemctl daemon-reload
systemctl restart promtail
systemctl status promtail

Install Loki:

$ mkdir -p /opt/app/{promtail,loki}
cat <
/opt/app/loki/loki.yaml
auth_enabled: false
server:
  http_listen_port: 3100
  grpc_listen_port: 9096
# (additional configuration omitted for brevity)
EOF
unzip loki-linux-amd64.zip
mv loki-linux-amd64 /opt/app/loki/loki
cat <
/etc/systemd/system/loki.service
[Unit]
Description=loki server
Wants=network-online.target
After=network-online.target
[Service]
ExecStart=/opt/app/loki/loki -config.file=/opt/app/loki/loki.yaml
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=loki
[Install]
WantedBy=default.target
EOF
systemctl daemon-reload
systemctl restart loki
systemctl status loki

Grafana Integration

In Grafana, add a new data source of type Loki with the URL http://loki:3100 , then use the Explore view to query logs, filter by labels, and visualize log streams.

Label‑Based Querying

Loki indexes only labels, enabling fast queries such as {job="apache"} |= "11.11.11.11" . Dynamic or high‑cardinality labels (e.g., per‑IP) can create many streams and impact performance, so label design should be considered carefully.

Performance Considerations

Avoid high‑cardinality labels to prevent excessive stream creation.

Chunk size and retention settings affect storage and query speed.

Logs must be ingested in chronological order; Loki rejects out‑of‑order data.

The article concludes with additional resources, interview questions, and community links.