Master Nightingale Alert Setup: From Email Media to Advanced Notification Rules

In daily operations we often encounter two extremes: an alarm storm that wakes you up at 3 am due to repeated alerts caused by network jitter, and silent failures where a system is down for half an hour without anyone noticing.

Tip: The problem is usually not a weak monitoring system but unreasonable alert configuration.

Nightingale, a domestic open‑source, enterprise‑grade monitoring system, offers a flexible alert engine and a user‑friendly interface, making it popular for building stable alerting solutions.

Below is a step‑by‑step tutorial from a frontline operations perspective, showing how to configure Nightingale’s core alert features to avoid false positives and missed alerts.

1. Alert Media – Email

Nightingale includes many built‑in alert media templates; this example demonstrates Email alerts. Other media can be found in the documentation.

Navigate: Notification → Notification Media → Search "Email"

SMTP Configuration

Server: SMTP address

Port: SMTP port

Username: Sending email address

Password: SMTP server login password

Sender: Name or alias displayed in the email

Tip:

If Nightingale cannot access the external network and needs a proxy, fill in the proxy address and port.

The sender name must not contain Chinese characters.

2. Notification Rules

Steps:

Notification → Notification Rules → Add

Basic Configuration

Name: Rule name

Authorized Team: Users in this team can manage or view the rule

Notification Configuration

Notification Media: Choose the media type (e.g., Email)

Message Template: Email alert content template

Recipients: Users who receive the alert emails

Recipient Team: Teams that receive the alert emails

Applicable Level: Alert level that triggers this rule

3. Alert Rules

Steps:

Add data source (refer to the previous article on data source integration)

Alert → Rule Management → Add

Basic Configuration

Rule Name: Name

Attachment Tags: Tags attached to all alert events generated by this rule (useful for later filtering)

Rule Configuration

Select alert data source type

Data Source Filter: Precisely match the data source selected in step 1

Built‑in Metric: PromQL expression for the alert

Trigger Alert: Choose the level that matches the notification rule

Execution Frequency: Frequency of the built‑in metric

Duration: How long the metric must stay true before triggering

Effective Configuration

Enable Immediately: Whether the rule is active

Effective Time: When the rule becomes effective

Notification Configuration

Notification Rule: Choose the rule configured above

Enable Recovery Notification: Whether to send recovery alerts

Repeat Notification Interval: Interval for repeated alerts if the issue persists

Maximum Send Count: 0 means no limit

Conclusion

Alert configuration is not a one‑time task; it requires continuous optimization. Nightingale provides strategy groups, multi‑level notifications, silence suppression, flexible templates, and more, offering a complete and controllable alert management solution. However, even the most powerful tool still depends on careful tuning by operations engineers.