Mastering Nightingale Monitoring: Architecture, Deployment Modes, and Best Practices

Nightingale has a simple architecture: a single binary can start the service for testing, but production requires MySQL and Redis. It is designed to handle multiple data centers, including edge sites with poor network quality.

Supported Data Sources

Initially supports Prometheus, VictoriaMetrics, and ElasticSearch for both visualization and alerting. Later versions focus on the alert engine, adding new data sources primarily for alerts.

Deployment Modes

Mode 1 : Monitoring data does not flow through Nightingale. Users handle data collection themselves and only configure the time‑series database in Nightingale for visualization and alerting. The architecture diagram shown illustrates this typical setup.

Mode 2 : Data flows through Nightingale. Categraf uses the remote‑write protocol to push data to Nightingale, which then forwards it to the configured time‑series stores via Pushgw.Writers in config.toml.

Single‑Node Production Mode

The n9e binary can be run directly after downloading from the Nightingale GitHub releases. Default port is 17000, username is root, and password is root.2020. The process depends on the etc and integrations directories, a database, and Redis, so MySQL and Redis connection details must be set in etc/config.toml.

[DB]
DBType = "mysql"
DSN = "user:pwd@tcp(localhost:3306)/n9e?charset=utf8mb4&parseTime=True&loc=Local"

[Redis]
Address = "127.0.0.1:6379"
Password = "pwd"
RedisType = "standalone"

Cluster Mode

Deploy the n9e process on multiple machines, ensuring identical configuration files and shared MySQL and Redis instances. Alert rules are automatically distributed among the instances; if one node fails, another takes over its rules.

Edge Mode

For scenarios with poor connectivity between central and edge data centers, deploy n9e‑edge in the edge site. It syncs alert rules from the central Nightingale, caches them locally, and evaluates alerts against the edge’s VictoriaMetrics. This ensures reliable alerting even if the edge loses connection to the central server.

Alert Architecture Overview

Central Nightingale handles alerts for Loki, ElasticSearch, and Prometheus in the main and edge‑A data centers.

Edge n9e‑edge handles alerts for VictoriaMetrics in edge‑B.

Conclusion

For operations teams, choosing a robust monitoring tool like Nightingale provides comprehensive system visibility and precise alerting, significantly improving fault response efficiency.