How to Integrate ELK with Zabbix for Real‑Time Log Alerting

1 Relationship between ELK and ZABBIX

ELK (Elasticsearch, Logstash, Kibana) is a log‑collection suite that can gather system, web, and application logs, filter and store them for real‑time search and analysis. By extracting error keywords (error, failed, warning, etc.) from logs and sending them to Zabbix, operators can receive immediate alerts for potential failures.

2 Using the Logstash‑Zabbix plugin

Logstash supports many output plugins; the community‑maintained logstash-output-zabbix plugin connects Logstash to Zabbix. Install it with:

/usr/share/logstash/bin/logstash-plugin install logstash-output-zabbix

Various logstash-plugin commands can list, update, or remove plugins.

2.1 List installed plugins

/usr/share/logstash/bin/logstash-plugin list

2.2 Install a plugin (example: kafka)

/usr/share/logstash/bin/logstash-plugin install logstash-output-kafka

2.3 Update plugins

/usr/share/logstash/bin/logstash-plugin update

2.4 Remove a plugin

/usr/share/logstash/bin/logstash-plugin remove logstash-output-kafka

3 Example configuration of logstash-output-zabbix

zabbix {
    zabbix_host => "[@metadata][zabbix_host]"
    zabbix_key => "[@metadata][zabbix_key]"
    zabbix_server_host => "x.x.x.x"
    zabbix_server_port => "xxxx"
    zabbix_value => "xxxx"
}

Parameters:

zabbix_host : name of the Zabbix host (required).

zabbix_key : item key in Zabbix.

zabbix_server_host : IP or hostname of the Zabbix server.

zabbix_server_port : listening port, default 10051.

zabbix_value : field whose content is sent to the Zabbix item, default “message”.

4 Integrating Logstash with Zabbix

The workflow reads logs, filters for keywords such as ERROR, Failed, WARNING, and forwards matching events to Zabbix.

4.1 Logstash pipeline configuration

Input (reading /var/log/secure):

input {
    file {
        path => "/var/log/secure"
        type => "system"
        start_position => "beginning"
    }
}

Filter (grok, mutate, date):

filter {
    grok {
        match => { "message" => "%{SYSLOGTIMESTAMP:message_timestamp} %{SYSLOGHOST:hostname} %{DATA:message_program}(?:\[%{POSINT:message_pid}\])?: %{GREEDYDATA:message_content}" }
    }
    mutate {
        add_field => ["[zabbix_key]","oslogs"]
        add_field => ["[zabbix_host]","Zabbix server"]
        remove_field => ["@version","message"]
    }
    date {
        match => [ "message_timestamp","MMM  d HH:mm:ss","MMM dd HH:mm:ss","ISO8601"]
    }
}

Output (Elasticsearch optional, Zabbix for alerts):

output {
    elasticsearch{
        index => "oslogs-%{+YYYY.MM.dd}"
        hosts => ["192.168.73.133:9200"]
        user => "elastic"
        password => "Goldwind@2019"
        sniffing => false
    }
    if [message_content] =~ /(ERR|error|ERROR|Failed)/ {
        zabbix {
            zabbix_host => "[zabbix_host]"
            zabbix_key => "[zabbix_key]"
            zabbix_server_host => "192.168.73.133"
            zabbix_server_port => "10051"
            zabbix_value => "message_content"
        }
    }
    #stdout { codec => rubydebug }
}

Save the configuration as file_to_zabbix.conf and start Logstash:

cd /usr/local/logstash
nohup bin/logstash -f config/file_to_zabbix.conf --path.data /tmp/ &

4.2 Zabbix side configuration

Create a template “logstash-output-zabbix”, an application set, and a monitoring item in the Zabbix web UI (screenshots omitted). Link the template to the monitored host (e.g., 192.168.73.135). When Logstash forwards a matching log line, Zabbix receives the message_content value.

4.3 Testing the alert

Generate a failed‑login entry in /var/log/secure and verify that Logstash filters the “Failed” keyword and that Zabbix shows the data in “Latest data”. Create a trigger that fires when the received value length > 0, and configure a DingTalk notification.

Summary

The architecture remains: Filebeat → Logstash → (optional Elasticsearch/Kibana) → Zabbix. The key component is the logstash-output-zabbix plugin, which enables real‑time log‑based alerts in Zabbix.