Promoter: Rendering AlertManager Graphs for DingTalk Notifications Using Go
The article introduces Promoter, a Go‑based webhook that fetches Prometheus metrics, renders alert graphs with gonum/plot, stores the images in S3‑compatible object storage, and embeds them in DingTalk notifications, providing deployment instructions, template customization, and core implementation details.
Previously a simple Python DingTalk receiver for AlertManager was used, but it required unstable web‑scraping to capture Prometheus graphs. The new approach, implemented in Go, directly renders the alert chart, uploads it to an S3‑compatible object store, and displays the image in DingTalk messages.
The default template resides at template/default.tmpl and can be customized to fit specific notification styles, referencing the open‑source project prometheus-webhook-dingtalk .
{{ define "__subject" }}[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] {{ .GroupLabels.SortedPairs.Values | join " " }} {{ if gt (len .CommonLabels) (len .GroupLabels) }}({{ with .CommonLabels.Remove .GroupLabels.Names }}{{ .Values | join " " }}{{ end }}){{ end }}{{ end }}
{{ define "__alertmanagerURL" }}{{ .ExternalURL }}/#/alerts?receiver={{ .Receiver }}{{ end }}
{{ define "default.__text_alert_list" }}{{ range . }}
### {{ .Annotations.summary }}
**详情:** {{ .Annotations.description }}
{{ range .Images }}
**条件:** `{{ .Title }}`

{{- end }}
**标签:**
{{ range .Labels.SortedPairs }}{{ if and (ne (.Name) "severity") (ne (.Name) "summary") }}> - {{ .Name }}: {{ .Value | markdown | html }}
{{ end }}{{ end }}
{{ end }}{{ end }}
{{ define "default.title" }}{{ template "__subject" . }}{{ end }}
{{ define "default.content" }}
{{ if gt (len .Alerts.Firing) 0 -}}
#### **{{ .Alerts.Firing | len }} 条报警**
{{ template "default.__text_alert_list" .Alerts.Firing }}
{{ range .AtMobiles }}@{{ . }}{{ end }}
{{- end }}
{{ if gt (len .Alerts.Resolved) 0 -}}
#### **{{ .Alerts.Resolved | len }} 条报警恢复**
{{ template "default.__text_alert_list" .Alerts.Resolved }}
{{ range .AtMobiles }}@{{ . }}{{ end }}
{{- end }}
{{- end }}The default configuration file ( /etc/promoter/config.yaml ) defines debug mode, HTTP port, Prometheus endpoint, metric resolution, S3 credentials, and DingTalk webhook settings. The service can be run via the Docker image cnych/promoter:v0.1.1 or deployed in Kubernetes using deploy/kubernetes/promoter.yaml .
debug: true
http_port: 8080
timeout: 5s
prometheus_url:
# Prometheus address
metric_resolution: 100
s3:
access_key:
secret_key:
endpoint: oss-cn-beijing.aliyuncs.com
region: cn-beijing
bucket:
dingtalk:
url: https://oapi.dingtalk.com/robot/send?access_token=
secret:
# secret for signatureAfter starting Promoter, configure AlertManager to point to the webhook URL (e.g., http://promoter.kube-mon.svc.cluster.local:8080/webhook ) in the receivers section.
route:
group_by: ['alertname','cluster']
group_wait: 30s
group_interval: 2m
repeat_interval: 1h
receiver: webhook
receivers:
- name: 'webhook'
webhook_configs:
- url: 'http://promoter.kube-mon.svc.cluster.local:8080/webhook' # configure promoter webhook
send_resolved: trueThe core implementation uses the Prometheus Go client to query metric ranges and the gonum.org/v1/plot library to draw charts. The Metrics function creates a client, performs a range query, and returns a matrix of samples. The PlotMetric function builds a plot, applies a color palette, draws lines for each series, adds a polygon for threshold visualization, and renders the final PNG canvas.
func Metrics(server, query string, queryTime time.Time, duration, step time.Duration) (promModel.Matrix, error) {
client, err := prometheus.NewClient(prometheus.Config{Address: server})
if err != nil {
return nil, fmt.Errorf("failed to create Prometheus client: %v", err)
}
api := prometheusApi.NewAPI(client)
value, _, err := api.QueryRange(context.Background(), query, prometheusApi.Range{Start: queryTime.Add(-duration), End: queryTime, Step: duration / step})
if err != nil {
return nil, fmt.Errorf("failed to query Prometheus: %v", err)
}
metrics, ok := value.(promModel.Matrix)
if !ok {
return nil, fmt.Errorf("unsupported result format: %s", value.Type().String())
}
return metrics, nil
}
func PlotMetric(metrics promModel.Matrix, level float64, direction string) (io.WriterTo, error) {
p, err := plot.New()
if err != nil {
return nil, fmt.Errorf("failed to create new plot: %v", err)
}
// ... (font setup, axis configuration, line drawing, polygon overlay, canvas rendering) ...
return c, nil
}For more details, refer to the GitHub repository cnych/promoter .
DevOps Cloud Academy
Exploring industry DevOps practices and technical expertise.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.