Why Choose Loki for Cloud‑Native Log Management? A Complete Deployment Guide

Why Use Loki

Loki is a lightweight log collection and analysis application that uses a promtail‑style agent to gather log data and store it in Loki, which can then be visualized and queried through Grafana as a datasource.

Loki's persistent storage supports Azure, GCS, S3, Swift, and local filesystems, with S3 and local being the most common; it also supports many log collection tools such as Logstash and Fluent Bit.

Advantages:

Supported clients include Promtail, Fluent Bit, Fluentd, Vector, Logstash, and Grafana Agent.

Promtail can ingest logs from multiple sources, including local files, systemd, Windows event logs, and Docker log drivers.

No required log format – JSON, XML, CSV, logfmt, or unstructured text are all accepted.

Log queries use the same syntax as metric queries.

Dynamic filtering and transformation of log lines during queries.

Easy calculation of metrics from logs.

Minimal indexing enables dynamic slicing and dicing of logs when new issues arise.

Cloud‑native support with Prometheus‑style scraping.

Simple comparison of log collection components:

Loki's Working Mechanism

When parsing logs, Loki primarily uses an index that contains timestamps and a subset of pod labels; the remaining data is the log content. Example query results are shown in the following diagrams.

{app="loki",namespace="kube-public"} is used as the index.

Log Collection Architecture

The official recommendation is to deploy Promtail as a DaemonSet on Kubernetes worker nodes, though other collectors can also be used.

Loki Deployment Modes

all (read/write mode)

All services run on a single node for both reads and writes.

read/write (read‑write separation mode)

In this mode, front‑end queries are forwarded to read nodes that host querier, ruler, and frontend, while write nodes host distributor and ingester.

Microservice mode

Each role runs as a separate process with its own configuration.

Server‑Side Deployment

Before deploying, ensure you have a Kubernetes cluster. The following sections show how to deploy Loki using the AllInOne mode.

AllInOne deployment mode

k8s deployment

The downloaded binary lacks a configuration file, so a complete AllInOne YAML file is provided below.

auth_enabled: false
target: all
ballast_bytes: 20480
server:
  grpc_listen_port: 9095
  http_listen_port: 3100
  graceful_shutdown_timeout: 20s
  grpc_listen_address: "0.0.0.0"
  grpc_listen_network: "tcp"
  grpc_server_max_concurrent_streams: 100
  grpc_server_max_recv_msg_size: 4194304
  grpc_server_max_send_msg_size: 4194304
  http_server_idle_timeout: 2m
  http_listen_address: "0.0.0.0"
  http_listen_network: "tcp"
  http_server_read_timeout: 30s
  http_server_write_timeout: 20s
  log_source_ips_enabled: true
  register_instrumentation: true
  log_format: json
  log_level: info
... (configuration continues) ...

Note: ingester.lifecycler.ring.replication_factor should be set to 1 for a single‑instance deployment. ingester.lifecycler.min_ready_duration defaults to 15s before the instance becomes ready. memberlist.node_name defaults to the host name if not set. memberlist.join_members must list all node hostnames/IPs in a multi‑instance setup; in k8s it can be bound to a StatefulSet service.

It is recommended to set query_range.results_cache.cache.enable_fifocache to false, though the example uses true. instance_interface_names defaults to ["en0","eth0"] and usually does not need modification.

Create a ConfigMap from the YAML file:

$ kubectl create configmap --from-file ./loki-all.yaml loki-all

Verify the ConfigMap creation with the following command (output shown in the image).

Create Persistent Storage

Persist Loki data using PersistentVolume (PV) and PersistentVolumeClaim (PVC). The example uses a hostPath backend.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: loki
  namespace: default
spec:
  hostPath:
    path: /glusterfs/loki
    type: DirectoryOrCreate
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteMany
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: loki
  namespace: default
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  volumeName: loki
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app: loki
  name: loki
  namespace: default
spec:
  podManagementPolicy: OrderedReady
  replicas: 1
  selector:
    matchLabels:
      app: loki
  template:
    metadata:
      annotations:
        prometheus.io/port: http-metrics
        prometheus.io/scrape: "true"
      labels:
        app: loki
    spec:
      containers:
      - args:
        - -config.file=/etc/loki/loki-all.yaml
        image: grafana/loki:2.5.0
        imagePullPolicy: IfNotPresent
        name: loki
        ports:
        - containerPort: 3100
          name: http-metrics
          protocol: TCP
        - containerPort: 9095
          name: grpc
          protocol: TCP
        - containerPort: 7946
          name: memberlist-port
          protocol: TCP
        resources:
          requests:
            cpu: 500m
            memory: 500Mi
          limits:
            cpu: 500m
            memory: 500Mi
        securityContext:
          readOnlyRootFilesystem: true
        volumeMounts:
        - mountPath: /etc/loki
          name: config
        - mountPath: /data
          name: storage
      restartPolicy: Always
      securityContext:
        fsGroup: 10001
        runAsGroup: 10001
        runAsNonRoot: true
        runAsUser: 10001
      serviceAccount: loki
      serviceAccountName: loki
      volumes:
      - emptyDir: {}
        name: tmp
      - name: config
        configMap:
          name: loki
      - persistentVolumeClaim:
          claimName: loki
        name: storage
---
kind: Service
apiVersion: v1
metadata:
  name: loki-memberlist
  namespace: default
spec:
  ports:
  - name: loki-memberlist
    protocol: TCP
    port: 7946
    targetPort: 7946
  selector:
    kubepi.org/name: loki
---
kind: Service
apiVersion: v1
metadata:
  name: loki
  namespace: default
spec:
  ports:
  - name: loki
    protocol: TCP
    port: 3100
    targetPort: 3100
  selector:
    kubepi.org/name: loki

Validate Deployment

When the pod shows a Running status, check the distributor via the API; it should display Active to confirm that logs are being correctly dispatched to the ingester.

Further deployment methods are omitted for brevity.