Choosing Between Prometheus and Zabbix: A Practical Guide to High‑Availability Monitoring

Introduction

The article is a transcript of Liu Yu’s live sharing on operational monitoring, focusing on the selection between Prometheus and Zabbix and practical implementations of monitoring solutions.

Agenda

Prometheus overview

Redis multi‑instance monitoring practice

Grafana integration of Zabbix and Prometheus

1. Prometheus Overview

Prometheus follows a pull‑based model: exporters expose metrics over HTTP, Prometheus scrapes them on a schedule, stores data locally in a time‑series database, and visualizes via its own UI or Grafana. It also supports a pushgateway for occasional push‑style metrics and an Alertmanager for flexible alerting.

Key reliability note: Prometheus tolerates failures and aims to minimize data loss, but it is not suited for systems requiring 100 % data accuracy.

2. High‑Availability (promeHA)

Because native Prometheus is single‑node, a custom HA layer was built using service registration. Core configuration snippets include:

etcdEndpoints=["127.0.0.1:2379"]
netCard="enp0s8"
vip="192.168.56.105"
num="2"
lock="/dev/prometheus"
cmds="/usr/local/prometheus/prometheus --config.file=/usr/local/prometheus/prometheus.yml"

The HA daemon handles three failure scenarios: Prometheus process crash, promeHA process crash, and node outage, automatically restarting processes or re‑electing a leader via VIP takeover.

3. Backend Storage

Prometheus’s default retention is 15 days, which may be insufficient. Remote storage was added using InfluxDB because it is easy to deploy, provides data expiration, SQL‑like queries, and fine‑grained permission control.

Remote write/read configuration examples:

Specify remote_write endpoint /api/v1/prom/write in Prometheus config.

Include authentication parameters when needed.

4. Prometheus Optimization

Two important tuning parameters: max_shards: controls the minimum number of shards for remote writes; Prometheus can increase shards automatically if needed. max_samples_per_send: limits the number of samples per remote‑write request to avoid overloading back‑ends.

5. Redis Monitoring

Redis exporters expose cache metrics. For multi‑instance setups, a single exporter can monitor several Redis instances via static or dynamic configurations.

Static configuration example (static_configs):

static_configs:
  - targets:
    - redis://redis-host-01:6379
    - redis://redis-host-02:6379

Dynamic discovery using file_sd_configs watches JSON files for target changes, reducing the need to restart Prometheus.

Consul‑based service discovery automates target registration. A custom collector syncs Redis metadata to Consul, and Prometheus watches Consul for changes. Example Consul registration JSON includes ID, name, address, tags, and checks. Prometheus scrape config uses consul_sd_configs and relabel_configs to extract the Redis master name and address.

6. Grafana Integration of Zabbix and Prometheus

Grafana plugins enable Zabbix as a data source. After installing the Zabbix plugin via grafana-cli plugins install, both Zabbix and Prometheus can be added as data sources, allowing unified dashboards that display Redis cache hit rates, expiration, evictions, network overhead, and other key metrics.

Dashboards combine Zabbix host groups, applications, and items with Prometheus queries, providing a single view for developers and ops engineers during high‑traffic events.

Conclusion

The sharing highlights practical challenges and solutions in building a resilient monitoring stack, encouraging readers to adapt the presented patterns to their own environments.