Master Filebeat: Collect Nginx, Elasticsearch, and MySQL Logs Seamlessly

Filebeat Overview

Filebeat is a lightweight log shipper that can forward and aggregate log files. It includes built‑in modules for common services such as Nginx, MySQL, Redis, Elasticsearch, and Logstash, simplifying collection, parsing, and visualization with a single command.

Installation and Configuration

Before installing Filebeat, ensure Elasticsearch and Kibana (version 7.6.2) are running.

Download the Filebeat 7.6.2 package from the official Elastic archive.

Extract the archive to a chosen directory.

Open Kibana (http://localhost:5601) and navigate to the Filebeat tutorial page.

Click the Add log data button to see the supported modules.

Open the Nginx module tutorial for detailed guidance.

Collect Nginx Logs

Enable the Nginx module and configure paths. filebeat modules enable nginx After enabling, the nginx.yml file in modules.d no longer has the disable suffix.

Edit nginx.yml to set the access and error log file paths.

# Module: nginx
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.6/filebeat-module-nginx.html

- module: nginx
  access:
    enabled: true
    var.paths: ["I:/developer/env/nginx-1.8.1/logs/access.log"]

  error:
    enabled: true
    var.paths: ["I:/developer/env/nginx-1.8.1/logs/error.log"]

Run filebeat setup to load index templates and dashboards. filebeat setup Start the Filebeat service. filebeat -e Verify the collected logs in Kibana’s Nginx logs dashboard.

Collect Elasticsearch Logs

Enable the Elasticsearch module and set log file locations. filebeat modules enable elasticsearch Edit elasticsearch.yml with the appropriate paths.

# Module: elasticsearch
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.6/filebeat-module-elasticsearch.html

- module: elasticsearch
  server:
    enabled: true
    var.paths: ["I:/developer/env/elasticsearch-7.6.2/logs/elasticsearch.log"]

  slowlog:
    enabled: true
    var.paths: ["I:/developer/env/elasticsearch-7.6.2/logs/elasticsearch_index_indexing_slowlog.log","I:/developer/env/elasticsearch-7.6.2/logs/elasticsearch_index_search_slowlog.log"]

  deprecation:
    enabled: true
    var.paths: ["I:/developer/env/elasticsearch-7.6.2/logs/elasticsearch_deprecation.log"]

Load templates and start Filebeat.

filebeat setup
filebeat -e

View the logs in Kibana’s Discover page.

Collect MySQL Logs

Enable the MySQL module and configure error and slow‑query logs. filebeat modules enable mysql Edit mysql.yml to point to the MySQL error and slow‑log files.

# Module: mysql
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.6/filebeat-module-mysql.html

- module: mysql
  error:
    enabled: true
    var.paths: ["C:/ProgramData/MySQL/MySQL Server 5.7/Data/DESKTOP-5NIMJ19.err"]

  slowlog:
    enabled: true
    var.paths: ["C:/ProgramData/MySQL/MySQL Server 5.7/Data/DESKTOP-5NIMJ19-slow.log"]

Load templates and start the service.

filebeat setup
filebeat -e

Open the MySQL logs dashboard in Kibana to inspect collected entries.

Conclusion

This article demonstrated how to use Filebeat to collect file‑based logs from Nginx, Elasticsearch, and MySQL, and highlighted that the same approach can be applied to other middleware services for convenient centralized logging.