Deploy a Full ELK Stack on a Single Ubuntu Server

Install Java Runtime

Update the package index and install OpenJDK 8 headless, which is required for Elasticsearch and Logstash.

$ apt update
$ apt install -y openjdk-8-jre-headless

Verify the installation:

$ java -version

Install Elasticsearch

$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
$ echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list
$ sudo apt update
$ sudo apt install -y elasticsearch=6.2.4
$ sudo systemctl daemon-reload
$ sudo systemctl enable elasticsearch.service

Install Kibana

$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
$ echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list
$ sudo apt update
$ sudo apt install -y kibana=6.2.4
$ sudo systemctl daemon-reload
$ sudo systemctl enable kibana.service

Install Logstash

If the package repository does not contain Logstash 6.2.4, download the .deb file directly and install it.

$ sudo apt install ./logstash-6.2.4.deb
$ sudo systemctl daemon-reload
$ sudo systemctl enable logstash.service

Full Installation Script

#!/bin/bash
apt update
apt install -y openjdk-8-jre-headless
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -
echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | tee -a /etc/apt/sources.list.d/elastic-6.x.list
apt update
apt install -y elasticsearch=6.2.4
apt install -y kibana=6.2.4
apt install -y ./logstash-6.2.4.deb
systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl enable logstash.service
systemctl enable kibana.service

Mount a Large Disk for Elasticsearch

Create a mount point and format the new 1 TB disk.

$ sudo mkdir /esdata
$ (echo n; echo p; echo 1; echo ; echo ; echo w) | sudo fdisk /dev/sdb
$ sudo mkfs -t ext4 /dev/sdb1
$ sudo mount /dev/sdb1 /esdata

Set ownership and permissions:

$ sudo chown elasticsearch:elasticsearch /esdata
$ sudo chmod 750 /esdata

Make the mount persistent by adding its UUID to /etc/fstab:

UUID=db048fa3-903b-4b85-a7ab-01c920283eeb /esdata ext4 defaults,nofail,barrier=0 0 2

Change Elasticsearch Data and Log Paths

# ----------------------------------- Paths ------------------------------------
path.data: /esdata
path.logs: /esdata

Configure Kibana

Allow remote access by setting server.host to 0.0.0.0 in /etc/kibana/kibana.yml.

#server.host: "localhost"
server.host: "0.0.0.0"

Configure Logstash

Create /etc/logstash/conf.d/beat2es.conf to receive beats on port 5044 and forward them to Elasticsearch.

input{
    beats{
        port => 5044
    }
}
output {
    elasticsearch {
        hosts => ["localhost:9200"]
        index => "beat-test-%{+YYYY.MM.dd}"
        sniffing => true
        template_overwrite => true
    }
}

Start Services

$ sudo systemctl start elasticsearch.service
$ sudo systemctl start kibana.service
$ sudo systemctl start logstash.service

Install Filebeat

$ curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.2.4-amd64.deb
$ sudo dpkg -i ./filebeat-6.2.4-amd64.deb
$ sudo systemctl daemon-reload
$ sudo systemctl enable filebeat.service

Verify the installation:

$ filebeat version
filebeat version 6.2.4 (amd64), libbeat 6.2.4

Configure Filebeat

Enable a log prospector and set the log file path:

- type: log
  enabled: true
  paths:
    - /home/nick/work/test.log

Send logs to Logstash:

output.logstash:
  hosts: ["your log server ip:5044"]

Enable multiline handling for stack traces:

multiline.pattern: '^\['
multiline.negate: true
multiline.match: after

Comment out the default Elasticsearch output:

#output.elasticsearch:
#  hosts: ["localhost:9200"]

Start Filebeat:

$ sudo systemctl start filebeat.service

Test the Setup

Append sample log lines to the test file to simulate an exception stack:

echo "[exception:]" >> work/test.log
echo "  at xxx" >> work/test.log
echo "  at xxx" >> work/test.log
echo "[OK]" >> work/test.log

Open Kibana in a browser, create an index pattern matching beat-test*, and verify that the multiline logs appear as a single event.

Summary

ELK is a heavyweight logging platform; this article shows how to set up a compact demo environment on a single Ubuntu server, covering installation, disk provisioning, configuration tweaks, and basic log ingestion with Filebeat.