Mastering Monitoring: From Basics to Advanced Zabbix Practices

Introduction

Monitoring is a critical component of operations and the entire product lifecycle, providing early fault detection and detailed post‑incident data for root‑cause analysis.

Monitoring Goals

Continuous real‑time monitoring : keep the system under constant observation.

Instant status feedback : know whether each component is normal, abnormal, or failed.

Reliability and safety assurance : ensure services run smoothly.

Business continuity : receive alerts immediately and resolve issues to maintain stable operations.

Monitoring Methods

Typical steps include:

Understand the monitoring target (e.g., CPU operation).

Define performance baseline metrics (CPU usage, load, user/kernel time, context switches, etc.).

Set alarm thresholds (what constitutes a fault).

Design fault‑handling procedures.

Core Monitoring Process

The four essential phases are:

Problem discovery : receive an alarm when a fault occurs.

Problem localization : analyse alarm details to pinpoint the cause (network, overload, firewall rule, etc.).

Problem resolution : address the issue according to its priority.

Problem summarization : document the cause and preventive measures to avoid recurrence.

Monitoring Tools Overview

MRTG – Multi Route Traffic Grapher, Perl‑based, uses SNMP to draw network traffic graphs.

Grnglia – high‑performance distributed monitoring system with RRDtool storage.

Cacti – PHP/MySQL/SNMP tool that creates graphs via RRDtool, supports templates and LDAP integration.

Nagios – enterprise‑grade service and host monitoring with alert notifications.

Smokeping – visualizes network latency, packet loss and other performance metrics using RRDtool.

OpenTSDB – time‑series database on HBase, stores raw metrics for long‑term analysis.

Zabbix – full‑stack distributed monitoring system, supports many protocols, agents, and rich templating.

Open‑Falcon (Xiaomi), OWL (TalkingData) and various third‑party SaaS solutions are mentioned as alternatives.

Zabbix Monitoring Workflow

Data collection : SNMP, Zabbix Agent, ICMP, SSH, IPMI, etc.

Data storage : typically MySQL, but other databases are supported.

Data analysis : historical graphs help pinpoint the root cause of incidents.

Data presentation : web UI (or custom mobile/Java/PHP front‑ends).

Alerting : phone, email, WeChat, SMS, with escalation mechanisms.

Alert handling : prioritize alerts (critical, important, etc.) and assign appropriate personnel.

Key Monitoring Metrics

Typical categories and example indicators:

Hardware : CPU, memory, disk, temperature, fan speed, voltage (IPMI, MegaCli). Zabbix IPMI Interface System : CPU load, context switches, user/kernel usage (70/30 rule), memory usage, swap, disk I/O, network I/O. Tools: htop, top, vmstat, iostat, iftop, sar. Zabbix Agent Interface Application : Nginx, PHP‑FPM, Redis, MySQL, RabbitMQ, etc. Zabbix Agent UserParameter, Zabbix JMX Interface, percona-monitoring-plulgins Network : latency, packet loss, bandwidth (Smokeping).

Traffic analysis : page views, source attribution (Piwik, Google Analytics, Baidu Tongji).

Log monitoring : system, application, network logs via ELK stack (Logstash, Elasticsearch, Kibana).

Security : firewall rules, WAF, vulnerability scanners, third‑party security services.

API : request methods (GET/POST/PUT/DELETE), availability, correctness, response time.

Performance : DNS response, HTTP connect time, page load time, element size ( Zabbix Web 监控).

Business : order rate, registration rate, active users, revenue, inventory, etc.

Alert Notification Channels

Common channels include SMS and email; phone calls and WeChat messages are also supported.

Interview Tips for Monitoring

A concise answer can cover:

Hardware monitoring via SNMP/IPMI.

System metrics such as CPU load, memory, disk and network I/O.

Service monitoring (Nginx, PHP‑FPM, MySQL, Redis, etc.) using built‑in status modules or custom scripts.

Network monitoring (latency, packet loss) with tools like Smokeping.

Security monitoring (firewalls, WAF, host hardening).

Web performance monitoring (page load, JS response).

Log collection and analysis (ELK stack).

Business‑level KPIs (order volume, user activity).

Traffic analysis (using analytics platforms or self‑hosted Piwik).

Visualization (dashboards, screen displays).

Automation via Zabbix active/passive modes and API integration.

Distributed monitoring concepts.

Conclusion

While many open‑source monitoring solutions exist, large‑scale enterprises often build custom platforms (e.g., Open‑Falcon, Sensu combined with InfluxDB and Grafana) to achieve full coverage and flexibility.