How to Visualize Nginx Access Logs with Loki and Grafana

Background

A client needed detailed website traffic statistics without using Google or Baidu analytics, so the only reliable source was the Nginx access logs. Exporting logs manually or writing custom scripts was impractical, prompting the adoption of a dedicated log‑monitoring stack.

Chosen Stack

The solution combines Loki for log storage, Promtail as the log shipper, and Grafana for visualization, all fed by Nginx. This lightweight stack is ideal for small‑to‑medium sites.

Installation of Promtail and Loki

Download the appropriate binary releases of Promtail and Loki, extract them, and start each with a configuration file.

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://localhost:3100/loki/api/v1/push

scrape_configs:
  - job_name: nginx
    pipeline_stages:
      - replace:
          expression: '(?:[0-9]{1,3}\.){3}([0-9]{1,3})'
          replace: '***'
    static_configs:
      - targets:
          - localhost
        labels:
          job: nginx_access_log
          host: expatsxxxxs
          agent: promtail
          __path__: /var/log/nginx/expatshxxxxs.access.log

Configure Nginx to Emit JSON Logs

Define a new log_format named json_analytics that outputs each request as a JSON object. The format includes fields such as timestamp, client IP, request details, upstream information, and GeoIP data.

log_format json_analytics escape=json '{'
    "msec": "$msec",
    "connection": "$connection",
    "connection_requests": "$connection_requests",
    "pid": "$pid",
    "request_id": "$request_id",
    "request_length": "$request_length",
    "remote_addr": "$remote_addr",
    "remote_user": "$remote_user",
    "remote_port": "$remote_port",
    "time_local": "$time_local",
    "time_iso8601": "$time_iso8601",
    "request": "$request",
    "request_uri": "$request_uri",
    "args": "$args",
    "status": "$status",
    "body_bytes_sent": "$body_bytes_sent",
    "bytes_sent": "$bytes_sent",
    "http_referer": "$http_referer",
    "http_user_agent": "$http_user_agent",
    "http_x_forwarded_for": "$http_x_forwarded_for",
    "http_host": "$http_host",
    "server_name": "$server_name",
    "request_time": "$request_time",
    "upstream": "$upstream_addr",
    "upstream_connect_time": "$upstream_connect_time",
    "upstream_header_time": "$upstream_header_time",
    "upstream_response_time": "$upstream_response_time",
    "upstream_response_length": "$upstream_response_length",
    "upstream_cache_status": "$upstream_cache_status",
    "ssl_protocol": "$ssl_protocol",
    "ssl_cipher": "$ssl_cipher",
    "scheme": "$scheme",
    "request_method": "$request_method",
    "server_protocol": "$server_protocol",
    "pipe": "$pipe",
    "gzip_ratio": "$gzip_ratio",
    "http_cf_ray": "$http_cf_ray",
    "geoip_country_code": "$geoip_country_code"
}' ;

Because the geoip_country_code variable is used, the Nginx GeoIP module must be compiled.

Enable GeoIP Support

Install the required libraries and recompile Nginx with the --with-http_geoip_module flag. yum -y install GeoIP GeoIP-data GeoIP-devel After recompilation, replace the old binary in the objs directory and reload Nginx with kill -USR2. The new JSON log format will then be active.

Sample JSON Log Entries

{"msec":"1633430998.322","connection":"4","connection_requests":"1","pid":"29887","request_id":"40770fec38c2e5a68714df5f7a67283d","request_length":"392","remote_addr":"106.19.96.55","remote_user":"","remote_port":"43746","time_local":"05/Oct/2021:18:49:58 +0800","time_iso8601":"2021-10-05T18:49:58+08:00","request":"GET / HTTP/2.0","request_uri":"/","args":"","status":"200","body_bytes_sent":"60949","bytes_sent":"61222","http_referer":"https://cn.bing.com/...","http_user_agent":"Mozilla/5.0 (iPhone; CPU iPhone OS 13_6 ...) Safari/604.1","http_x_forwarded_for":"","http_host":"www.expatsholidays.com","server_name":"www.expatsholidays.com","request_time":"0.003","upstream":"127.0.0.1:9000","upstream_connect_time":"0.000","upstream_header_time":"0.002","upstream_response_time":"0.002","upstream_response_length":"60991","upstream_cache_status":"","ssl_protocol":"TLSv1.2","ssl_cipher":"ECDHE-RSA-AES128-GCM-SHA256","scheme":"https","request_method":"GET","server_protocol":"HTTP/2.0","pipe":".","gzip_ratio":"","http_cf_ray":"","geoip_country_code":""}
{... another entry ...}

Deploy Grafana via Docker

Run Grafana in a container, then log in with the default credentials ( admin/admin) and reset the password. docker run -d -p 3000:3000 grafana/grafana After login, add Loki as a data source.

Use the Explore view to confirm that logs are being ingested.

Import a Dashboard

Import a pre‑built dashboard by ID. The default dashboard lacks a world map, so install the grafana-worldmap-panel plugin.

grafana-cli plugins install grafana-worldmap-panel

Restart Grafana and reload the dashboard.

The map panel now displays, though external map tiles may be blocked; a reverse proxy can resolve this.

Conclusion

By combining Nginx, Promtail, Loki, and Grafana, you can provide clients with a real‑time, searchable view of website traffic without relying on third‑party analytics services.