Configuring Argo CD Webhooks, Prometheus Metrics, and DingTalk Notifications

This article introduces how to optimize Argo CD application detection with webhooks, monitor Argo CD using Prometheus, and receive application status change notifications via DingTalk.

Webhook Configuration

Argo CD polls the Git repository every three minutes to detect changes; to eliminate this delay you can configure the API server to receive webhook events, enabling real‑time detection. Argo CD supports webhooks from GitHub, GitLab, Bitbucket, Bitbucket Server, and Gogs. The example below shows how to configure a GitLab webhook.

In the GitLab project http://git.k8s.local/course/devops-demo-deploy add a webhook with the URL http://argocd.k8s.local/api/webhook . The optional secret token can be any string. Because a self‑signed HTTPS certificate is used, disable SSL verification.

Then add the same secret token to the Argo CD secret:

➜  ~ kubectl edit secret argocd-secret -n argocd
apiVersion: v1
kind: Secret
metadata:
  name: argocd-secret
  namespace: argocd
type: Opaque
data:
...
stringData:
  # gitlab webhook secret
  webhook.gitlab.secret: youdianzhishi

After saving, the change takes effect immediately. Test the webhook from GitLab and check the Argo CD API server pod logs; a successful push event will be logged.

➜  ~ kubectl logs -f argocd-server-5cc96b75b4-zws2c -n argocd
time="2021-07-08T07:15:32Z" level=info msg="finished streaming call with code OK" ...
time="2021-07-08T07:15:37Z" level=info msg="Received push event repo: http://git.k8s.local/course/devops-demo-deploy, revision: master, touchedHead: true"
time="2021-07-08T07:15:37Z" level=info msg="Requested app 'devops-demo' refresh"

Metrics

Argo CD exposes two sets of Prometheus metrics, making it easy to integrate with monitoring and alerting systems.

The default metrics endpoint is argocd-metrics:8082/metrics , which provides application health, sync status, and sync history metrics.

API request/response metrics (request count, response codes, etc.) are available at argocd-server-metrics:8083/metrics .

To scrape these metrics, add the annotation prometheus.io/scrape: "true" to the corresponding Service objects. Example for the metrics service:

➜  ~ kubectl edit svc argocd-metrics -n argocd
apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/scrape: "true"
  ...

Similarly, annotate the server‑metrics and repo‑server services (port 8083 and 8084 respectively) with the same annotation.

After annotation, Prometheus will automatically discover the metrics endpoints. If you use the Prometheus Operator, you can create a ServiceMonitor object instead.

Import the official Argo CD Grafana dashboard from GitHub to visualize the metrics.

Message Notification

While Alertmanager can handle alerts, you may also want to send application sync status to a specific channel. Argo CD does not have built‑in sync notifications, but you can integrate with third‑party systems such as Slack, SMTP, Telegram, Discord, or DingTalk.

ArgoCD Notifications – integrates with many notification services.

Argo Kube Notifier – generic Kubernetes controller for sending notifications.

Kube Watch – posts notifications to Slack, HipChat, Mattermost, etc.

Argo CD provides resource hooks (PreSync, Sync, PostSync, SyncFail) that can run scripts. To send a notification after a successful sync, you can use a simple curl command, but for richer integration we use ArgoCD Notifications with DingTalk.

Download the ArgoCD Notifications manifests:

➜  ~ wget https://raw.githubusercontent.com/argoproj-labs/argocd-notifications/stable/manifests/install.yaml

Create a DingTalk robot in a group and choose the keyword‑based authentication method, using the keyword ArgoCD .

Edit install.yaml and add the DingTalk webhook configuration to the argocd-notifications-cm ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-notifications-cm
data:
  service.webhook.dingtalk: |
    url: https://oapi.dingtalk.com/robot/send?access_token=YOUR_TOKEN
    headers:
      - name: Content-Type
        value: application/json
  context: |
    argocdUrl: http://argocd.k8s.local
  template.app-sync-change: |
    webhook:
      dingtalk:
        method: POST
        body: |
          {"msgtype": "markdown","markdown": {"title":"ArgoCD Sync Status","text": "### ArgoCD Sync Status\n> - app name: {{.app.metadata.name}}\n> - sync status: {{ .app.status.operationState.phase}}\n> - time: {{.app.status.operationState.startedAt}}\n> - URL: [Open ArgoCD]({{.context.argocdUrl}}/applications/{{.app.metadata.name}}?operation=true) \n"}}
  trigger.on-deployed: |
    - description: Application is synced and healthy. Triggered once per commit.
      oncePer: app.status.sync.revision
      send: [app-sync-change]
      when: app.status.operationState.phase in ['Succeeded'] and app.status.health.status == 'Healthy'
  # ... other triggers ...
  subscriptions: |
    - recipients: [dingtalk]
      triggers: [on-sync-running, on-deployed, on-sync-failed, on-sync-succeeded]

Note: the recipients field should be webhook:dingtalk in some versions; adjust if necessary.

You can also add conditional subscriptions, for example sending alerts to Slack or email based on specific triggers.

After editing the ConfigMap, apply the manifest:

➜  ~ kubectl apply -f install.yaml
➜  ~ kubectl get pods -n argocd
NAME                                           READY   STATUS    RESTARTS   AGE
argocd-application-controller-0                1/1     Running   0          5d4h
argocd-dex-server-76ff776f97-ds7mm            1/1     Running   0          5d4h
argocd-notifications-controller-5c548f8dc9-dx824   1/1     Running   0          9m22s
argocd-redis-747b678f89-w99wf                  1/1     Running   0          5d4h
argocd-repo-server-6fc4456c89-586zl            1/1     Running   0          5d4h
argocd-server-5cc96b75b4-zws2c                 1/1     Running   0          4d22h

Trigger a Git change to start the GitOps pipeline; you should receive a DingTalk message. If not, debug with the CLI:

➜  ~ kubectl exec -it argocd-notifications-controller-5c548f8dc9-dtq7h -n argocd -- /app/argocd-notifications template notify app-sync-change guestbook --recipient dingtalk
DEBU[0000] Sending request: POST /robot/send?access_token=YOUR_TOKEN HTTP/1.1
Host: oapi.dingtalk.com
Content-Type: application/json

{"msgtype":"markdown","markdown":{"title":"ArgoCD Sync Status","text":"### ArgoCD Sync Status\n> - app name: guestbook\n> - sync status: Succeeded\n> - time:2021-07-03T12:53:44Z\n> - URL: [Open ArgoCD](http://argocd.k8s.local/applications/guestbook?operation=true) \n"}}
DEBU[0000] Received response: HTTP/2.0 200 OK
...{"errcode":0,"errmsg":"ok"}

For more details, refer to the official ArgoCD Notifications documentation: https://argocd-notifications.readthedocs.io/en/stable/ .