Master ELK Stack: Install & Configure Elasticsearch, Logstash, Kibana

ELK Overview

ELK stands for Elasticsearch, Logstash, and Kibana. Elasticsearch provides real‑time full‑text search and analytics, Logstash collects and transforms logs, and Kibana offers a web UI for visualizing data stored in Elasticsearch.

Environment

Two CentOS 6.5 machines (IP 192.168.1.202 and 192.168.1.201) are used. Required components: elasticsearch, logstash, kibana, nginx, httpd, and redis.

Installation

Import the Elasticsearch GPG key and add the yum repository:

# rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
# vim /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-5.x]
name=Elasticsearch repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Install Elasticsearch: # yum install -y elasticsearch Install Java (1.8 or newer) and verify the installation:

# wget http://download.oracle.com/otn-pub/java/jdk/8u131-b11/.../jdk-8u131-linux-x64.rpm
# rpm -ivh jdk-8u131-linux-x64.rpm
# java -version

Elasticsearch Configuration

Create a data directory and set ownership:

# mkdir -p /data/es-data
# chown -R elasticsearch:elasticsearch /data/es-data

Adjust log directory ownership:

# chown -R elasticsearch:elasticsearch /var/log/elasticsearch/

Edit /etc/elasticsearch/elasticsearch.yml to set cluster name, node name, data and log paths, memory lock, network host, HTTP port, and CORS settings:

cluster.name: demon
node.name: elk-1
path.data: /data/es-data
path.logs: /var/log/elasticsearch/
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9200
http.cors.enabled: true
http.cors.allow-origin: "*"

Start the service and adjust JVM memory if needed:

# /etc/init.d/elasticsearch start
# vim /etc/elasticsearch/jvm.options
-Xms512m
-Xmx512m
# /etc/init.d/elasticsearch restart

Enable the service at boot:

# chkconfig elasticsearch on

Important System Settings

Increase file descriptor and process limits for the ELK user:

# vim /etc/security/limits.conf
elk soft nofile 65536
elk hard nofile 65536
elk soft nproc 2048
elk hard nproc 2048
elk soft memlock unlimited
elk hard memlock unlimited

Adjust /etc/security/limits.d/90-nproc.conf to set soft nproc 2048. Disable the system‑call filter if bootstrap checks fail:

# vim /etc/elasticsearch/elasticsearch.yml
bootstrap.system_call_filter: false

Logstash Installation and Usage

Install Logstash and create a shortcut:

# rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
# yum install -y logstash
# ln -s /usr/share/logstash/bin/logstash /bin/

Run a simple pipeline to test input/output:

# logstash -e 'input { stdin { } } output { stdout {} }'

Example configuration file ( /etc/logstash/conf.d/elk.conf) to read system, secure, HTTP, and Nginx logs and send them to Elasticsearch:

input {
  file { path => "/var/log/messages" type => "system" start_position => "beginning" }
  file { path => "/var/log/secure" type => "secure" start_position => "beginning" }
  file { path => "/var/log/httpd/access_log" type => "http" start_position => "beginning" }
  file { path => "/usr/local/nginx/logs/elk.access.log" type => "nginx" start_position => "beginning" }
}
output {
  if [type] == "system" {
    elasticsearch { hosts => ["192.168.1.202:9200"] index => "nagios-system-%{+YYYY.MM.dd}" }
  }
  if [type] == "secure" {
    elasticsearch { hosts => ["192.168.1.202:9200"] index => "nagios-secure-%{+YYYY.MM.dd}" }
  }
  if [type] == "http" {
    elasticsearch { hosts => ["192.168.1.202:9200"] index => "nagios-http-%{+YYYY.MM.dd}" }
  }
  if [type] == "nginx" {
    elasticsearch { hosts => ["192.168.1.202:9200"] index => "nagios-nginx-%{+YYYY.MM.dd}" }
  }
}

Plugin Installation

Install the elasticsearch‑head plugin via Docker or source:

# docker run -p 9100:9100 mobz/elasticsearch-head:5
# OR
# yum install -y npm
# git clone https://github.com/mobz/elasticsearch-head.git
# cd elasticsearch-head
# npm install
# npm run start

Kibana Installation and Usage

Download and extract Kibana 5.4.0, then configure:

# wget https://artifacts.elastic.co/downloads/kibana/kibana-5.4.0-linux-x86_64.tar.gz
# tar -xzf kibana-5.4.0-linux-x86_64.tar.gz
# mv kibana-5.4.0-linux-x86_64 /usr/local
# ln -s /usr/local/kibana-5.4.0-linux-x86_64 /usr/local/kibana
# vim /usr/local/kibana/config/kibana.yml
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://192.168.1.202:9200"
kibana.index: ".kibana"
# yum -y install screen
# screen
# /usr/local/kibana/bin/kibana

Access Kibana at http://<em>IP</em>:5601.

ELK Practical Usage

Define JSON log formats for Nginx and Apache, then update Logstash pipelines to ingest those logs. Example Nginx log_format json and Apache LogFormat definitions are provided in the original guide.

Redis Integration

Install Redis, enable daemon mode, bind to the ELK host, and start the service:

# yum install -y redis
# vim /etc/redis.conf
daemonize yes
bind 192.168.1.202
# /etc/init.d/redis restart
# redis-cli -h 192.168.1.202 info

Use Logstash to push data into Redis:

input { stdin { } }
output {
  redis {
    host => "192.168.1.202"
    port => "6379"
    password => "test"
    db => "1"
    data_type => "list"
    key => "elk-test"
  }
}

Read from Redis and forward to Elasticsearch:

input {
  redis {
    host => "192.168.1.202"
    port => "6379"
    password => "test"
    db => "6"
    data_type => "list"
    key => "nagios_system"
    batch_count => 1
  }
}
output {
  elasticsearch { hosts => ["192.168.1.202:9200"] index => "nagios-system-%{+YYYY.MM.dd}" }
}

Deploy ELK

Classify logs (system, access, error, device, debug) and standardize them as JSON where possible. Collect logs in the order: system → error → runtime → access.

Delete old indices to free space, e.g.:

curl -X DELETE http://xx.xx.com:9200/logstash-*-`date +%Y-%m-%d -d "-30 days"`

References

Original article: https://www.cnblogs.com/yuhuLin/p/7018858.html