Enable Prometheus Monitoring for TDMQ Pulsar: Step‑by‑Step Guide

What is Prometheus?

Prometheus is an open‑source, full‑stack monitoring solution designed for containers and micro‑services. It stores data as time‑series, offers high‑precision collection (1–5 seconds), fast deployment, rich plugins, and integrates well with Grafana for visualisation.

Fine‑grained data collection (1–5 s precision).

Quick cluster deployment and script creation.

Fast open‑source updates and extensive language support.

Mathematical model enables complex business‑logic monitoring.

Beautiful Grafana dashboards.

Deep integration with container monitoring.

Why integrate Pulsar with Prometheus?

TDMQ Pulsar Professional clusters have been commercialised for over a year, gaining many users. Large‑scale customers often run their own unified monitoring systems and want to avoid the Tencent Cloud console. By exposing metrics in Prometheus format, users can feed data to their own Prometheus servers, enable Grafana dashboards, and drive workload HPA for automated operations.

Design Options

Two monitoring solutions are offered:

Existing Tencent Cloud integrated monitoring (console + observability platform alerts).

New Prometheus‑based monitoring, which differs in data collection path and metric aggregation.

The original cloud‑native solution aggregates metrics on the Barad platform before displaying them, resulting in longer latency. The Prometheus solution opens a direct VPC‑level Exporter endpoint, allowing users to scrape metrics directly, reducing latency but requiring users to perform aggregation themselves.

Usage Guide

Log in to the TDMQ console.

In the left navigation, select **Cluster Management**, click the target cluster ID to open the instance details page.

Open the **Monitoring** sheet.

Click the **Prometheus Monitoring** entry in the top‑right corner to obtain the monitoring target, then choose the appropriate network type and configuration.

Submit to receive a set of monitoring targets.

Edit the prometheus.yml file to add a node_exporter scrape job.

Access the visualization interface to view the configured metrics.

Prometheus Configuration Example

job_name: prometheus-tdmq-pulsar
honor_timestamps: true
scrape_interval: 60s
metrics_path: /tencent-cloud-metrics/
scheme: http
static_configs:
  - targets:
    - 10.x.x.x:10001
    - 10.x.x.x:10002

Key fields:

honor_timestamps : When true, Prometheus uses the timestamps provided by the Exporter instead of the server’s receipt time.

scrape_interval : Frequency of metric collection (default is one minute).

metrics_path : URL path for metric retrieval; set to /tencent-cloud-metrics/.

scheme : Protocol used to access the Exporter; currently only http is supported.