How to Collect Nginx Access and Error Logs with Filebeat, Logstash, and Rsyslog

Because Nginx is widely used as a high‑performance HTTP and reverse‑proxy server, its access and error logs are essential for user behavior analysis and security monitoring. This article demonstrates several ways to collect those logs efficiently using Filebeat, Logstash, and rsyslog.

1. Directly send logs from Filebeat to Elasticsearch

Configure filebeat.yml to read /usr/local/nginx/logs/*.log and output to Elasticsearch.

- type: log
  enabled: true
  paths:
    - /usr/local/nginx/logs/*.log

After setting the Elasticsearch hosts in the output.elasticsearch section, start Filebeat:

./filebeat -e -c filebeat.yml -d "publish"

Use the Elasticsearch‑head plugin or Kibana to verify that both access.log and error.log entries appear in the filebeat‑* index.

2. Send logs from Filebeat to Logstash, then to Elasticsearch

Create filebeat-pipeline.conf for Logstash:

input {
  beats {
    port => "5044"
  }
}
output {
  elasticsearch { hosts => ["172.28.65.24:9200"] }
  stdout { codec => rubydebug }
}

Start Logstash with automatic config reload:

bin/logstash -f filebeat-pipeline.conf --config.reload.automatic

Modify filebeat.yml to disable the Elasticsearch output and enable output.logstash pointing to the Logstash host and port 5044, then restart Filebeat. Logs will flow through Logstash, allowing preprocessing before reaching Elasticsearch.

3. Forward Nginx logs via rsyslog to Logstash, then to Elasticsearch

When the target server cannot run Filebeat, configure Nginx or rsyslog to emit syslog messages.

Option A: Nginx syslog output

access_log syslog:server=172.28.65.32:514,facility=local7,tag=nginx_access_log,severity=info;
error_log syslog:server=172.28.65.32:514,facility=local7,tag=nginx_error_log,severity=info;

Reload Nginx to apply the changes.

Option B: rsyslog file monitoring

$IncludeConfig /etc/rsyslog.d/*.conf

# In /etc/rsyslog.d/nginx-log.conf
$ModLoad imfile
$InputFilePollInterval 1
$WorkDirectory /var/spool/rsyslog
$PrivDropToGroup adm

$InputFileName /usr/local/nginx/logs/access.log
$InputFileTag nginx-access:
$InputFileStateFile stat-nginx-access
$InputFileSeverity info
$InputRunFileMonitor

$InputFileName /usr/local/nginx/logs/error.log
$InputFileTag nginx-error:
$InputFileStateFile stat-nginx-error
$InputFileSeverity error
$InputRunFileMonitor

*.* @172.28.65.32:514

Restart rsyslog with systemctl restart rsyslog. Logstash receives the syslog on port 514 using a pipeline like:

input {
  syslog {
    type => "system-syslog"
    port => 514
  }
}
output {
  elasticsearch { hosts => ["172.28.65.24:9200"] index => "system-syslog-%{+YYYY.MM}" }
  stdout { codec => rubydebug }
}

Start Logstash:

bin/logstash -f syslog-pipeline.conf --config.reload.automatic

Visiting the Nginx web service (e.g., http://172.28.65.32/) now produces log entries visible in both Logstash’s console and Elasticsearch/Kibana.

These three approaches allow flexible log collection depending on the environment and processing requirements.