Step-by-Step Guide: Deploy ELK Stack on CentOS for Powerful Log Analysis

Experiment Environment

Set up two Elasticsearch nodes (node1 and node2) and an Apache server on a CentOS network.

Environment Preparation

Install required packages: Elasticsearch RPM, Kibana RPM, Logstash RPM, Node.js, and PhantomJS.

elasticsearch-5.5.0.rpm
kibana-5.5.1-x86_64.rpm
logstash-5.5.1.rpm
node-v8.2.1.tar.gz
phantomjs-2.1.1-linux-x86_64.tar.bz2

Deploy Elasticsearch

On both nodes, verify Java, install Elasticsearch, edit /etc/elasticsearch/elasticsearch.yml to set cluster name, node name, data and log paths, network host, and enable CORS. Reload systemd, enable and start the service, create data and log directories, set ownership, and verify the service is listening on port 9200.

# java -version
# rpm -ivh elasticsearch-5.5.0.rpm
# vim /etc/elasticsearch/elasticsearch.yml
# systemctl daemon-reload
# systemctl enable elasticsearch.service
# systemctl start elasticsearch
# netstat -nultp | grep 9200

Check Cluster Health

Access http://192.168.192.113:9200 to view node information, then query /_cluster/health?pretty and /_cluster/state?pretty to ensure the cluster status is green.

Install Elasticsearch‑head Plugin

Install Node.js and PhantomJS, extract elasticsearch-head.tar.gz, run npm install, and start the plugin on port 9100.

# yum install -y gzip
# tar -zxf node-v8.2.1.tar.gz
# cd node-v8.2.1 && ./configure && make && make install
# tar -jxf phantomjs-2.1.1-linux-x86_64.tar.bz2 -C /usr/local/src/
# cp phantomjs /usr/local/bin
# tar -zxf elasticsearch-head.tar.gz
# cd elasticsearch-head && npm install
# npm run start &

Install and Configure Logstash

Install Logstash RPM, enable and start the service, then create system.conf to collect system logs and send them to Elasticsearch.

# rpm -ivh logstash-5.5.1.rpm
# systemctl enable logstash.service
# systemctl start logstash.service
# vim /etc/logstash/conf.d/system.conf
input { file { path => "/var/log/messages" type => "system" start_position => "beginning" } }
output { elasticsearch { hosts => ["192.168.192.113:9200"] index => "system-%{+YYYY.MM.dd}" } }

Install Kibana

Install Kibana RPM, enable the service, and edit /etc/kibana/kibana.yml to point to the Elasticsearch endpoint. Access Kibana at http://192.168.192.113:5601.

# rpm -ivh kibana-5.5.1-x86_64.rpm
# systemctl enable kibana
# vim /etc/kibana/kibana.yml
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://192.168.192.113:9200"

Add Indices and Apache Logs

Create a test index with curl -XPUT, configure Logstash to read Apache access and error logs, and start Logstash with the new configuration.

# curl -XPUT 'localhost:9200/index-demo/test/1?pretty' -H 'Content-Type: application/json' -d '{"user":"zhangsan","mesg":"hello world"}'
# vim /etc/logstash/conf.d/apache_log.conf
input { file { path => "/var/log/httpd/access_log" type => "access" start_position => "beginning" } file { path => "/var/log/httpd/error_log" type => "error" start_position => "beginning" } }
output { if [type] == "access" { elasticsearch { hosts => ["192.168.192.113:9200"] index => "apache_access-%{+YYYY.MM.dd}" } } if [type] == "error" { elasticsearch { hosts => ["192.168.192.113:9200"] index => "apache_error-%{+YYYY.MM.dd}" } } }

Summary

The tutorial walks through deploying the ELK stack on CentOS, covering installation, configuration, health verification, plugin setup, and integration of system and Apache logs, providing a complete, visualizable log analysis solution.