How to Monitor Consul Server with Prometheus on Kubernetes: Step‑by‑Step Guide

In a micro‑service architecture, a Consul Server cluster acts as the core service‑discovery hub, and Prometheus can provide full‑visibility monitoring and intelligent alerts for it.

Enable metrics in Consul

global:
  metrics:
    enabled: true
    enableAgentMetrics: true

Upgrade the Consul Server cluster

$ helm -n consul upgrade consul -f consul-value.yaml consul
Release "consul" has been upgraded. Happy Helming!
NAME: consul
LAST DEPLOYED: Wed Jul  2 08:54:24 2025
NAMESPACE: consul
STATUS: deployed
REVISION: 4
NOTES:
Thank you for installing HashiCorp Consul!

To learn more about the release, run:
  $ helm status consul --namespace consul
  $ helm get all consul --namespace consul

Consul on Kubernetes Documentation:
https://www.consul.io/docs/platform/k8s

Consul on Kubernetes CLI Reference:
https://www.consul.io/docs/k8s/k8s-cli

Verify the metrics endpoint

$ curl -s 10.244.58.206:8500/v1/agent/metrics?token=dfa12077-35ad-b1b2-98f9-9213ff52e20d&format=prometheus | head
# HELP consul_acl_ResolveToken This measures the time it takes to resolve an ACL token.
# TYPE consul_acl_ResolveToken summary
consul_acl_ResolveToken{quantile="0.5"} 0.03899800032377243
consul_acl_ResolveToken{quantile="0.9"} 0.1492609977722168
consul_acl_ResolveToken{quantile="0.99"} 0.1492609977722168
consul_acl_ResolveToken_sum 1275.3871084274724
consul_acl_ResolveToken_count 1218
# HELP consul_acl_authmethod_delete 
# TYPE consul_acl_authmethod_delete summary
consul_acl_authmethod_delete{quantile="0.5"} NaN

Create a Prometheus ServiceMonitor

cat <<'EOF' | kubectl apply -f -
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: consul-server
  namespace: consul
  labels:
    release: monitor
spec:
  namespaceSelector:
    matchNames:
    - consul
  selector:
    matchLabels:
      release: consul
      component: server
  endpoints:
  - port: http
    path: /v1/agent/metrics
    params:
      format: ["prometheus"]
      token: ["dfa12077-35ad-b1b2-98f9-9213ff52e20d"]
EOF

Validate collection in Grafana

Add Grafana dashboard

Import the Grafana dashboard with ID 10890 to visualize Consul metrics.

Conclusion

Monitoring is not just for reacting to incidents; it enables proactive detection and intervention. Fine‑grained Prometheus monitoring of a Consul cluster improves observability and lays a solid foundation for automated operations and elastic scaling.