Step‑by‑Step Guide to Building an ELK Stack on CentOS 6.7

Environment: a CentOS 6.7 machine with IP 192.168.88.250. Install JDK 1.8.0_77 in /usr/local/java and add the following to /etc/profile:

export JAVA_HOME=/usr/local/java/jdk1.8.0_77
export PATH=$JAVA_HOME/bin:$PATH

Reload the profile with source /etc/profile and verify with java -version.

1. Install ElasticSearch 2.1.0:

wget https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.1.0/elasticsearch-2.1.0.tar.gz
tar xf elasticsearch-2.1.0.tar.gz
cd /usr/local/elasticsearch-2.1.0
./plugin -install mobz/elasticsearch-head
./elasticsearch -Des.insecure.allow.root=true

Test the node with curl -XGET 192.168.88.250:9200 and you should see a JSON response containing the cluster name and version.

2. Install NGINX (1.7.8) and configure it as a reverse proxy for Kibana and as a JSON log collector:

# Example nginx.conf snippets
worker_processes 1;
events { worker_connections 1024; }
http {
    log_format json '{"@timestamp":"$time_iso8601","host":"$server_addr","clientip":"$remote_addr","size":$body_bytes_sent,"responsetime":$request_time,"upstreamtime":"$upstream_response_time","upstreamhost":"$upstream_addr","http_host":"$host","url":"$uri","xff":"$http_x_forwarded_for","referer":"$http_referer","agent":"$http_user_agent","status":"$status"}';
    access_log /var/log/nginx/access.log_json json;
    server {
        listen 80;
        server_name kibana_server;
        location / {
            root /var/www/kibana;
            index index.html index.htm;
        }
        location ~ ^/kibana4/.* {
            proxy_pass http://kibana4;
            rewrite ^/kibana4/(.*) /$1 break;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $host;
            auth_basic "Restricted";
            auth_basic_user_file /etc/nginx/conf.d/kibana.myhost.org.htpasswd;
        }
    }
}

3. Install Logstash 2.1.1 and create stdin.conf to read the NGINX JSON logs and ship them to ElasticSearch:

wget https://download.elastic.co/logstash/logstash/logstash-2.1.1.tar.gz
tar xf logstash-2.1.1.tar.gz
cd /usr/local/logstash-2.1.1/bin
vim stdin.conf   # write the following configuration
input {
  file {
    path => "/var/log/nginx/access.log_json"
    codec => "json"
  }
}
filter {
  mutate { split => ["upstreamtime", ","] }
  mutate { convert => {"upstreamtime" => "float"} }
}
output {
  elasticsearch {
    hosts => ["192.168.88.250:9200"]
    index => "logstash-%{type}-%{+YYYY.MM.dd}"
    document_type => "%{type}"
    workers => 1
    flush_size => 20000
    idle_flush_time => 10
    template_overwrite => true
  }
}
./logstash -f stdin.conf &

4. Install Kibana 4.3.1, edit config/kibana.yml to point to the ElasticSearch URL, and start it:

wget https://download.elastic.co/kibana/kibana/kibana-4.3.1-linux-x64.tar.gz
tar xf kibana-4.3.1-linux-x64.tar.gz
cd /usr/local/kibana-4.3.1-linux-x64
vim ./config/kibana.yml   # set elasticsearch.url to http://192.168.88.250:9200
./kibana &

After all components are running, open http://192.168.88.250:5601 in a browser. If the Discover view shows no data, ensure the time filter is set at least eight hours ahead because Logstash timestamps are in UTC.

Use the Add/Remove fields controls in Discover to customize which log fields are displayed, and explore the Visualize tab to create custom charts.