Step-by-Step Guide to Deploying an ELK Stack (Elasticsearch, Logstash, Kibana) with Filebeat on Multiple Linux Servers
This guide details the preparation of three Linux servers, installation and configuration of JDK, Elasticsearch, Kibana, Filebeat, and Logstash, creation of necessary users and directories, and the commands to start each component, culminating in a functional ELK logging platform.
The deployment environment consists of three servers (192.168.236.179, .180, .181) each intended to run Elasticsearch, Logstash, Kibana, Filebeat, and JDK.
JDK installation and system tuning :
tar xf jdk-1.8.0_linux-x64_bin.tar.gz -C /usr/local/vim /etc/profile.d/jdk.sh(add environment variables)
export JAVA_HOME=/usr/local/jdk-1.8.0/export PATH=$PATH:$JAVA_HOME/binexport CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jarsource /etc/profile.d/jdk.shVerify with java -version.
Elasticsearch installation on the first node (es1) :
tar xf elasticsearch-7.6.2-linux-x86_64.tar.gz -C /usr/local/mv elasticsearch-7.6.2 elasticsearchEdit /usr/local/elasticsearch/config/elasticsearch.yml to set:
cluster.name: my-elknode.name: node-1path.data: /data/es/datapath.logs: /data/es/logbootstrap.memory_lock: truenetwork.host: 192.168.236.179http.port: 9200discovery.seed_hosts: ["192.168.236.179", "192.168.236.180", "192.168.236.181"]cluster.initial_master_nodes: ["192.168.236.179", "192.168.236.180", "192.168.236.181"]discovery.zen.minimum_master_nodes: 2gateway.recover_after_nodes: 2.
Create data and log directories and a dedicated user:
mkdir -pv /data/es/datamkdir -pv /data/es/logsuseradd elasticchown -R elastic:elastic /data/es/chown -R elastic:elastic /usr/local/elasticsearch/.
Start Elasticsearch service as the elastic user:
su - elasticcd /usr/local/elasticsearchnohup ./bin/elasticsearch > /tmp/elastic.log &Check with netstat -nltp | grep -E "9200|9300" and curl http://192.168.236.179:9200/, which returns a JSON block showing node name, cluster name, version, etc.
Copy the Elasticsearch directory to the other two nodes:
scp -r elasticsearch [email protected]:/usr/local/scp -r elasticsearch [email protected]:/usr/local/.
Kibana installation :
tar xf kibana-7.6.2-linux-x86_64.tar.gz -C /usr/local/ln -sv kibana-7.6.2-linux-x86_64/ kibanaEdit /usr/local/kibana/config/kibana.yml to set:
server.port: 5601server.host: "172.16.150.159"elasticsearch.hosts: ["http://172.16.150.157:9200"]i18n.locale: "zh-CN". Start Kibana: nohup ./kibana --allow-root > /tmp/kibana.log &.
Filebeat installation :
tar xf filebeat-7.6.2-linux-x86_64.tar.gz -C /usr/local/cd /usr/local/filebeatEdit filebeat.yml as needed, then start:
nohup /usr/local/filebeat/filebeat -e -c /usr/local/filebeat/filebeat.yml > /tmp/filebeat.log &.
Install Nginx and configure its access logs to output JSON format (details omitted).
Logstash installation :
tar xf logstash-7.6.2.tar.gz -C /usr/local/cd /usr/local/logstash/config/Create logstash_nginx.conf with the following sections:
input { file { path => "/opt/nginx/logs/access.log" type => "nginxlog" start_position => "beginning" } }filter { if [type]=="nginxlog" { grok { match => ["message", "%{COMMONAPACHELOG}"] } date { match => ["timestamp","dd/MMM/yyyy:HH:mm:ss Z"] } } }output { stdout { codec => rubydebug } elasticsearch { hosts => "192.168.236.179:9200","192.168.236.180:9200","192.168.236.181:9200" } }Start Logstash:
nohup /usr/local/logstash/bin/logstash -f logstash_nginx.conf > /tmp/logstch.log &.
After all components are running, access the Kibana UI to verify that logs are being ingested and visualized. The article notes that this setup can be extended to a distributed logging system using ELK + Filebeat + Kafka.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
