Master Prometheus: Installation, Configuration, PromQL Basics, and Grafana Integration

Prometheus Overview

Rapid business growth requires a 24/7 monitoring system that can collect real‑time metrics, trigger alerts, and provide data for performance optimisation.

Why monitor?

Real‑time data collection and timely alerts enable quick problem resolution and performance tuning.

Four monitoring elements

Monitoring target (host, service, resource, URL)

Monitoring method (pull or push)

Monitoring schedule (7×24, 5×8)

Alert recipients and severity levels

Technology selection

mrtg – SNMP‑based traffic graphs

cacti – PHP‑based SNMP collector

ntop – network traffic analysis

nagios – plugin‑rich alerting

centreon – Nagios integration

ganglia – lightweight large‑scale monitoring

open‑falcon – high‑efficiency monitoring

zabbix – popular multi‑platform solution

prometheus – cloud‑native time‑series monitoring

Analysis shows that Prometheus best fits the requirements.

Prometheus Usage

Architecture and Ecosystem

Prometheus consists of:

Prometheus Server – stores time‑series data and evaluates rules.

Retrieval component – pulls metrics from exporters or Pushgateway.

Service discovery – dynamically discovers scrape targets.

TSDB (Time‑Series Database) – the storage engine.

HTTP server – exposes the API and UI.

Experiment Environment Planning

prometheus10 – runs the Prometheus server.

mysql11 – runs MySQL, node_exporter, and mysqld_exporter (monitoring database + host).

application12 – runs a Spring Boot application and node_exporter (monitoring Java app + host).

Install Prometheus Server

Upload prometheus-2.29.1.linux-amd64.tar.gz to /opt/software.

Extract to /opt/module and rename the directory to /opt/module/prometheus-2.29.1.

Inspect the default prometheus.yml configuration.

Start Prometheus:

# cd /opt/module/prometheus-2.29.1
./prometheus --web.enable-lifecycle

Prometheus UI is reachable at http://<em>prometheus10</em>:9090.

Monitor Linux Host with node_exporter

Upload node_exporter-1.2.2.linux-amd64.tar.gz to /opt/software.

Extract to /opt/module and rename to /opt/module/node_exporter-1.2.2.

Run the exporter:

# cd /opt/module/node_exporter-1.2.2
./node_exporter

Metrics are exposed at http://<em>host</em>:9100/metrics.

Add a scrape job to prometheus.yml:

scrape_configs:
  - job_name: "node_exporter"
    static_configs:
      - targets: ["mysql11:9100"]

Reload Prometheus (e.g., curl -X POST http://localhost:9090/-/reload).

Monitor MySQL with mysqld_exporter

Create an exporter user in MySQL:

CREATE USER 'exporter'@'localhost' IDENTIFIED BY '123456' WITH MAX_USER_CONNECTIONS 3;
GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'exporter'@'localhost';
FLUSH PRIVILEGES;

Upload mysqld_exporter-0.13.0.linux-amd64.tar.gz to /opt/software, extract to /opt/module, and rename to /opt/module/mysqld_exporter-0.13.0.

Create my.cnf in the exporter directory:

[client]
user=exporter
password=123456

Start the exporter:

# cd /opt/module/mysqld_exporter-0.13.0
./mysqld_exporter --config.my-cnf=my.cnf

Metrics are available at http://<em>mysql11</em>:9104/metrics.

Add a scrape job to prometheus.yml:

- job_name: "mysql_exporter"
    static_configs:
      - targets: ["mysql11:9104"]

Monitor Java Spring Boot Application

Add Actuator and Micrometer Prometheus dependencies to pom.xml:

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

Enable the Prometheus endpoint in application.yml (or application.properties):

management:
  server:
    port: 8091
  endpoint:
    prometheus:
      enabled: true
  endpoints:
    web:
      exposure:
        include: health,info,prometheus

Build and run the jar: java -jar springboot-prometheus.jar Metrics are exposed at http://<em>application12</em>:8091/actuator/prometheus.

Add a scrape job to prometheus.yml:

- job_name: "springboot_exporter"
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ["application12:8091"]

PromQL Introduction

Basic Usage

Instant vector query: prometheus_http_requests_total Label matching: prometheus_http_requests_total{instance="localhost:9090"} Regex matching:

mysql_global_status_buffer_pool_pages{state=~"data|free"}

Range Queries

Select a time window with square brackets, e.g.

mysql_global_status_buffer_pool_pages{state=~"data|free"}[5m]

.

Offset, Aggregation, and Operators

Use offset 5m to shift the evaluation time. Aggregation functions such as sum(), avg(), max() operate on vectors. Boolean modifiers ( bool) turn comparisons into true/false series.

PromQL Operators

Mathematical: +, -, *, /, %, ^ Boolean: ==, !=, >, <, >=, <= (use bool to get scalar results).

Set operators: and, or, unless for combining vectors.

Aggregation functions: sum(), min(), max(), avg(), stddev(), stdvar(), count(), count_values(), bottomk(), topk(), quantile(). Use by or without to control label grouping, e.g.

sum(rate(node_cpu_seconds_total{mode!="idle"}[5m])) by (instance)

.

Grafana Simple Use

Install Grafana

Upload grafana-enterprise-8.1.2.linux-amd64.tar.gz to /opt/software.

Extract to /opt/module.

# tar xzvf /opt/software/grafana-enterprise-8.1.2.linux-amd64.tar.gz -C /opt/module

Start Grafana:

# cd /opt/module/grafana-8.1.2
./bin/grafana-server web > ./grafana.log 2>&1 &

Grafana UI is reachable at http://<em>host</em>:3000 (default credentials: admin / admin).

Integrate Prometheus

In Grafana, add a new data source of type Prometheus and set the URL to http://<em>prometheus10</em>:9090. Test the connection and save.

Create Dashboards

Dashboards can be built manually by adding panels and writing PromQL queries, or imported from JSON files available on https://grafana.com/dashboards. Imported dashboards are added via the Grafana UI (Dashboard → Import).