Step‑by‑Step Guide: Deploying an ELK Log Analysis Stack on CentOS

Deploy ELK Stack on CentOS

Provides a complete, step‑by‑step tutorial for installing and configuring Elasticsearch, Logstash, and Kibana on a CentOS environment.

1. Experiment Environment

Two Elasticsearch nodes (node1 and node2) and an Apache server are used for log collection.

2. Environment Preparation

Required packages:

elasticsearch-5.5.0.rpm v8.2.1.tar.gz
kibana-5.5.1-x86_64.rpm
elasticsearch-head.tar.gz
node-v8.2.1.tar.gz
phantomjs-2.1.1-linux-x86_64.tar.bz2
logstash-5.5.1.rpm

Disable firewall and SELinux, then update /etc/hosts with node IPs.

systemctl stop firewalld && systemctl disable firewalld
setenforce 0
vim /etc/hosts
192.168.192.113 node1
192.168.192.114 node2
192.168.192.116 apache

3. Deploy Elasticsearch

Node1

Check Java version; install OpenJDK 8 if missing.

[root@node1 ~]# java -version
openjdk version "1.8.0_412"

Install the RPM, then edit /etc/elasticsearch/elasticsearch.yml:

cluster.name: my-elk-cluster
node.name: node1
path.data: /data/elk_data
path.logs: /var/log/elk_logs
network.host: 0.0.0.0
http.port: 9200
discovery.zen.ping.unicast.hosts: ["node1","node2"]
http.cors.enabled: true
http.cors.allow-origin: "*"

Enable the service, create data and log directories, set ownership, start Elasticsearch, and verify the port.

systemctl daemon-reload
systemctl enable elasticsearch.service
mkdir -p /data/elk_data /var/log/elk_logs
chown elasticsearch:elasticsearch /data/elk_data /var/log/elk_logs
systemctl start elasticsearch
netstat -nultp | grep 9200

Node2

Repeat the same steps, changing node.name to node2 and updating the hosts file accordingly.

4. Verify Cluster

Access http://192.168.192.113:9200 to view node information. Check cluster health (should be green) and cluster state:

curl http://192.168.192.113:9200/_cluster/health?pretty
curl http://192.168.192.113:9200/_cluster/state?pretty

5. Install Elasticsearch‑head Plugin

Install Node.js and PhantomJS, then extract and run the plugin.

yum install -y gzip
tar zxf node-v8.2.1.tar.gz
cd node-v8.2.1 && ./configure && make && make install
tar -jxf phantomjs-2.1.1-linux-x86_64.tar.bz2 -C /usr/local/src/
cp /usr/local/src/phantomjs-2.1.1-linux-x86_64/bin/phantomjs /usr/local/bin/
cd /root && tar -zxf elasticsearch-head.tar.gz && cd elasticsearch-head
npm install
npm run start &

Access the UI at http://192.168.192.113:9100 to view cluster health and indices.

6. Create an Index

Use curl to add a test document to index-demo:

curl -XPUT 'localhost:9200/index-demo/test/1?pretty' -H 'Content-Type: application/json' -d '{"user":"zhangsan","mesg":"hello world"}'

7. Install Logstash

Install the RPM, ensure Java is available, start the service, and create a symbolic link for convenience.

rpm -ivh logstash-5.5.1.rpm
systemctl start logstash.service
ln -s /usr/share/logstash/bin/logstash /usr/local/bin/

Configure Logstash to read system logs and forward them to Elasticsearch.

input {
  file { path => "/var/log/messages" type => "system" start_position => "beginning" }
}
output {
  elasticsearch { hosts => ["192.168.192.113:9200"] index => "system-%{+YYYY.MM.dd}" }
}

systemctl restart logstash

8. Install Kibana

Install the RPM, enable the service, and configure /etc/kibana/kibana.yml:

systemctl enable kibana
vim /etc/kibana/kibana.yml
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://192.168.192.113:9200"
kibana.index: ".kibana"

Access the dashboard at http://192.168.192.113:5601 to visualize logs.

9. Add Apache Logs

Create a Logstash configuration to ingest Apache access and error logs.

input {
  file { path => "/var/log/httpd/access_log" type => "access" start_position => "beginning" }
  file { path => "/var/log/httpd/error_log" type => "error" start_position => "beginning" }
}
output {
  if [type] == "access" {
    elasticsearch { hosts => ["192.168.192.113:9200"] index => "apache_access-%{+YYYY.MM.dd}" }
  }
  if [type] == "error" {
    elasticsearch { hosts => ["192.168.192.113:9200"] index => "apache_error-%{+YYYY.MM.dd}" }
  }
}

Start Logstash with the new configuration using a helper script.

#!/bin/bash
/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/apache_log.conf

Conclusion

The tutorial demonstrates end‑to‑end deployment of the ELK stack on CentOS, covering installation, configuration, index creation, and log visualization, providing a solid foundation for log‑driven monitoring and analysis.