Build a Full Grafana‑Prometheus Monitoring Stack for MySQL, RabbitMQ, Redis & TiDB

1. Overview of Grafana

Grafana is a cross‑platform open‑source metric analysis and visualization tool that supports flexible client charts, multiple panel plugins (heatmap, line, etc.), various data sources (Graphite, InfluxDB, Prometheus, Elasticsearch, CloudWatch, KairosDB), alert notifications via Slack or PagerDuty, mixed data source displays, event annotations, and ad‑hoc filters.

1.1 Grafana Features

Display: Fast, flexible client charts with many panel plugins.

Data Sources: Supports Graphite, InfluxDB, OpenTSDB, Prometheus, Elasticsearch, CloudWatch, KairosDB, etc.

Alerting: Define alert rules, send notifications to Slack, PagerDuty, etc.

Mixed Views: Combine different data sources in one chart.

Annotations: Hover over events to see full metadata.

Filters: Ad‑hoc filters applied to all queries of a data source.

Grafana provides a versatile monitoring interface with email alerts.

1.2 Prometheus vs Zabbix

Both are open‑source monitoring frameworks. Prometheus is more modular and flexible (separate alerting, exporter, and server components) and uses a pull model, while Zabbix is monolithic, uses a push model, and requires installing agents that read logs or databases.

2. Prometheus Installation and Configuration

Download and extract:

wget https://github.com/prometheus/prometheus/releases/download/v2.7.2/prometheus-2.7.2.linux-amd64.tar.gz

tar xvfz prometheus-2.7.2.linux-amd64.tar.gz

Run Prometheus:

cd prometheus-2.7.2.linux-amd64

./prometheus --config.file=prometheus.yml

The default prometheus.yml contains global settings ( scrape_interval: 15s, evaluation_interval: 15s) and a scrape job for the Prometheus server itself.

2.1 Add Node Exporter (machine metrics)

wget https://github.com/prometheus/node_exporter/releases/download/v0.17.0/node_exporter-0.17.0.linux-amd64.tar.gz

tar xvfz node_exporter-0.17.0.linux-amd64.tar.gz

cd node_exporter-0.17.0.linux-amd64

./node_exporter &

Verify with curl http://<em>host</em>:9100/metrics.

2.2 Configure Prometheus to scrape Node Exporter

scrape_configs:

- job_name: 'server'

static_configs:

- targets: ['<em>host</em>:9100']

Reload Prometheus; the target should show as UP in /targets.

3. Grafana Installation and Setup

wget https://dl.grafana.com/oss/release/grafana-6.0.0.linux-amd64.tar.gz

tar -zxvf grafana-6.0.0.linux-amd64.tar.gz

Start Grafana:

cd grafana-6.0.0

./bin/grafana-server web &

Default login is admin/admin; change the password on first login.

3.1 Add Plugins

./grafana-cli plugins install grafana-piechart-panel

Restart Grafana after installing plugins.

3.2 Add Prometheus Data Source

In Grafana UI, go to Configuration → Data Sources → Add data source → Prometheus . Set the URL to http://<em>prometheus_host</em>:9090 and click Save & Test .

3.3 Import Dashboards

Use the + → Import menu, enter a dashboard ID (e.g., 8919 for a generic system dashboard) or upload a JSON file, then select the Prometheus data source.

4. Email Alert Configuration

Edit defaults.ini in the Grafana conf directory to set SMTP settings (e.g., smtp.qq.com:465 for QQ mail). Restart Grafana and create alert rules in dashboards; alerts will be sent via email when thresholds are crossed.

5. MySQL Monitoring

5.1 Install MySQL (if needed)

wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm

rpm -ivh mysql-community-release-el7-5.noarch.rpm

yum install mysql-server -y

chown -R root:root /var/lib/mysql

service mysqld restart

mysql -u root

UPDATE mysql.user SET authentication_string=PASSWORD('123456') WHERE User='root';

5.2 Install mysqld_exporter

wget https://github.com/prometheus/mysqld_exporter/releases/download/v0.10.0/mysqld_exporter-0.10.0.linux-amd64.tar.gz

tar -xvf mysqld_exporter-0.10.0.linux-amd64.tar.gz

cat > .my.cnf <<EOF

[client]

user=root

password=root

EOF

./mysqld_exporter -config.my-cnf=.my.cnf &

5.3 Add MySQL target to Prometheus

- job_name: 'mysql'

static_configs:

- targets: ['<em>mysql_host</em>:9104']

Port 9104 is fixed for the MySQL exporter.

6. Grafana HTTP API (API Keys)

Grafana’s admin HTTP API requires Basic Auth; API keys are linked to organizations and roles (Viewer, Editor, Admin). Create an API key via the UI, then use it as a Bearer token in the Authorization header for API calls (e.g., creating or deleting dashboards).

6.1 Create API Key

Navigate to Configuration → API Keys → Add API Key , choose a role, set an optional expiration ( secondsToLive), and save. Use the generated key in requests: Authorization: Bearer <em>YOUR_KEY</em>.

6.2 Delete API Key

List keys via GET /api/auth/keys, note the id, then delete with DELETE /api/auth/keys/{id}.

7. RabbitMQ Monitoring

7.1 Install rabbitmq_exporter

wget https://github.com/kbudde/rabbitmq_exporter/releases/download/v1.0.0-RC5/rabbitmq_exporter-1.0.0-RC5.linux-amd64.tar.gz

tar -xvf rabbitmq_exporter-1.0.0-RC5.linux-amd64.tar.gz

cd rabbitmq_exporter-1.0.0-RC5.linux-amd64

RABBIT_USER=guest RABBIT_PASSWORD=guest OUTPUT_FORMAT=JSON PUBLISH_PORT=9099 RABBIT_URL=http://localhost:15672 nohup ./rabbitmq_exporter &

7.2 Add RabbitMQ target to Prometheus

- job_name: 'RabbitMQ'

static_configs:

- targets: ['<em>rabbitmq_host</em>:9099']

7.3 Add Prometheus data source in Grafana (same as before) and import dashboard ID 2121.

8. Redis Monitoring

8.1 Install redis_exporter

wget https://github.com/oliver006/redis_exporter/releases/download/v1.0.3/redis_exporter-v1.0.3.linux-amd64.tar.gz

tar -xvf redis_exporter-v1.0.3.linux-amd64.tar.gz

8.2 Run exporter

No password:

./redis_exporter redis://<em>redis_host</em>:6379 &

With password:

./redis_exporter -redis.addr <em>redis_host</em>:6379 -redis.password <em>YOUR_PASSWORD</em> &

8.3 Add Redis target to Prometheus

- job_name: 'redis'

static_configs:

- targets: ['<em>redis_host</em>:9121']

8.4 Import Redis dashboard (ID 731) in Grafana.

9. TiDB Monitoring

9.1 Install node_exporter (used for TiDB metrics)

wget https://github.com/prometheus/node_exporter/releases/download/v0.15.2/node_exporter-0.15.2.linux-amd64.tar.gz

tar -xzf node_exporter-0.15.2.linux-amd64.tar.gz

cd node_exporter-0.15.2.linux-amd64

./node_exporter --web.listen-address=":9100" --log.level=info &

9.2 Add TiDB target to Prometheus

- job_name: 'tidb'

honor_labels: true

static_configs:

- targets: ['<em>tidb_host</em>:10080']

Reload Prometheus; the TiDB target should appear UP.

9.3 Import TiDB dashboard (tidb.json) in Grafana.

10. Final Remarks

After configuring all exporters and Prometheus scrape jobs, verify each target’s UP status in http://<em>prometheus_host</em>:9090/targets. Then, in Grafana, import the corresponding dashboards (MySQL, RabbitMQ, Redis, TiDB) and set up alert rules as needed. The complete stack provides end‑to‑end observability for servers and services.