Step-by-Step Installation and Configuration of Prometheus, Node Exporter, and Grafana on a Kubernetes Cluster

1. Install and configure Prometheus on the master node (192.168.210.85)

Extract the archive and move it to /data/prometheus :

tar -zxvf prometheus-2.23.0.linux-amd64.tar.gz -C /data

mv prometheus-2.23.0.linux-amd64 /data/prometheus

Set ownership:

chown prometheus:prometheus -R /data/prometheus

Create a systemd service file ( /etc/systemd/system/prometheus.service ) with the following content:

[Unit] Description=Prometheus After=network.target [Service] ExecStart=/data/prometheus/prometheus --config.file=/data/prometheus/prometheus.yml --storage.tsdb.path=/data/prometheus/data User=prometheus [Install] WantedBy=multi-user.target

Enable and start the service:

systemctl enable prometheus

systemctl start prometheus

Verify that Prometheus is listening on port 9090:

netstat -nltp|grep prometheus

2. Install node_exporter on the master and worker nodes

On the master node:

tar -zxvf node_exporter-1.0.1.linux-amd64.tar.gz -C /usr/local/

mv /usr/local/node_exporter-1.0.1.linux-amd64 /usr/local/node_exporter

Run it in the background:

nohup /usr/local/node_exporter/node_exporter &

Repeat the same steps on k8s-node1 (192.168.210.177) (see the detailed commands in the original text).

Update prometheus.yml to scrape the new targets:

static_configs: - targets: ['192.168.210.85:9090'] - job_name: 'k8s-node1' - targets: ['192.168.210.177:9100'] - job_name: 'k8s-master1' - targets: ['192.168.210.85:9100']

Restart Prometheus to apply the changes:

systemctl restart prometheus

3. Install and configure Grafana on node2 (192.168.210.195)

Download and extract Grafana:

wget https://dl.grafana.com/oss/release/grafana-7.3.5.linux-amd64.tar.gz

tar -zxvf grafana-7.3.5.linux-amd64.tar.gz

Move it to /usr/local/ and create a symlink:

mv grafana-7.3.5 /usr/local/

ln -s /usr/local/grafana-7.3.5/ /usr/local/grafana

Create a non‑login user for Grafana and set up data directories:

useradd -s /sbin/nologin -M grafana

mkdir -p /data/grafana

Adjust ownership:

chown -R grafana:grafana /usr/local/grafana

chown -R grafana:grafana /data/grafana

Edit /usr/local/grafana/conf/defaults.ini (or a copy) to point to the data directories:

data = /data/grafana/data logs = /data/grafana/log plugins = /data/grafana/plugins provisioning = /data/grafana/conf/provisioning

Create a systemd service file ( /etc/systemd/system/grafana-server.service ) with the following content:

[Unit] Description=Grafana After=network.target [Service] User=grafana Group=grafana Type=notify ExecStart=/usr/local/grafana/bin/grafana-server -homepath /usr/local/grafana Restart=on-failure [Install] WantedBy=multi-user.target

Start and enable the service:

systemctl start grafana-server

Verify it is listening on port 3000:

netstat -nltp|grep grafana

Access Grafana via a browser at http://192.168.210.195:3000 (default login: admin / admin).

4. Verify basic monitoring of the master and nodes

Open the Prometheus targets page ( http://192.168.210.85:9090/targets ) to see the status of the master, node1, and node_exporter instances.

Similarly, view Grafana dashboards to monitor the collected metrics.