Deploy ELK Stack and Scale a Web App on Kubernetes: Step‑by‑Step Guide

Architecture Overview

The setup consists of two pods: an ELK pod and a web‑application pod. The ELK pod exposes Logstash on port 5044 (for Filebeat) and Kibana on port 5601 (for user access). The web pod exposes a service for browser access.

Deploying ELK

1. SSH into a machine with kubectl access.

2. Create elkhost.yaml:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: elkhost
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: elkhost
    spec:
      containers:
      - name: elkhost
        image: sebp/elk:622
        tty: true
        ports: [{"containerPort": 5601}, {"containerPort": 5044}]

Apply it with kubectl create -f elkhost.yaml and verify the pod:

root@host# kubectl get pods
NAME               READY   STATUS    RESTARTS   AGE
elkhost-xxxxxx     1/1     Running   0          3m

3. Expose Kibana via NodePort (port 30001) with elkkibana-svc.yaml:

apiVersion: v1
kind: Service
metadata:
  name: elkkibana
spec:
  type: NodePort
  ports:
  - port: 5601
    nodePort: 30001
  selector:
    name: elkhost

4. Expose Logstash via ClusterIP (port 5044) with elkhost-svc.yaml:

apiVersion: v1
kind: Service
metadata:
  name: elkhost
spec:
  type: ClusterIP
  ports:
  - port: 5044
    targetPort: 5044
  selector:
    name: elkhost

Apply both services and verify:

root@host# kubectl create -f elkhost-svc.yaml && kubectl create -f elkkibana-svc.yaml
service "elkhost" created
service "elkkibana" created
root@host# kubectl get service
NAME       TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
elkhost    ClusterIP  10.43.103.244   <none>        5044/TCP         9s
elkkibana  NodePort   10.43.219.137   <none>        5601:30001/TCP   9s

5. Access Kibana at http://<node‑IP>:30001.

Deploying the Web Application

Create elkwebdemo.yaml for the web pod:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: elkwebdemo
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: elkwebdemo
    spec:
      containers:
      - name: elkwebdemo
        image: bolingcavalry/elkdemo:0.0.1-SNAPSHOT
        tty: true
        ports:
        - containerPort: 8080

Apply it with kubectl create -f elkwebdemo.yaml and verify the pod.

Expose the web service via NodePort (port 30002) using elkwebdemo-svc.yaml:

apiVersion: v1
kind: Service
metadata:
  name: elkwebdemo
spec:
  type: NodePort
  ports:
  - port: 8080
    nodePort: 30002
  selector:
    name: elkwebdemo

Apply and check the service list.

Scaling the Web Application

Scale the deployment to three replicas:

kubectl scale deployment elkwebdemo --replicas=3

Verify the three pods are running and generate traffic by repeatedly accessing http://<node‑IP>:30002/hello/tom. The logs from all three pods appear in Kibana under the “host” field.

Viewing Logs in Kibana

Open Kibana, navigate to “Discover”, and you will see logs collected from the three web pods as well as ELK components.