How to Expose Ingress Metrics for Prometheus Monitoring in Kubernetes

Preface

Recently a client needed to monitor ingress traffic. After researching, the deployment was successful and the process and issues are recorded here.

Expose Ingress Monitoring Port

By default nginx‑ingress exposes metrics on port 10254 at path

/metrics

. To make the port reachable, edit the Service definition to add a listener named

https-metrics

on port 10254, and modify the deployment to open the same port in the pod as

metrics

.

<code>spec:
  type: ClusterIP
  ports:
  - name: https-webhook
    port: 443
    targetPort: webhook
  - name: https-metrics
    port: 10254
    targetPort: 10254
</code>

<code>ports:
- name: http
  containerPort: 80
  protocol: TCP
- name: https
  containerPort: 443
  protocol: TCP
- name: webhook
  containerPort: 8443
  protocol: TCP
- name: metrics
  containerPort: 10254
  protocol: TCP
</code>

Static Scrape Configuration

Add a custom scrape job via the

additionalScrapeConfigs

field in

prometheus-prometheus.yaml

. Create

prometheus-additional.yaml

with the following content:

<code>- job_name: nginx-ingress
  metrics_path: /metrics
  scrape_interval: 5s
  static_configs:
  - targets:
    - 172.16.200.102:10254
    - 172.16.200.103:10254
    - 172.16.200.104:10254
</code>

Create a secret that contains this file:

<code>$ kubectl create secret generic ingress-nginx-additional-configs --from-file=./prometheus-additional.yaml -n monitoring
</code>

Reference the secret in

prometheus-prometheus.yaml

:

<code>serviceAccountName: prometheus-k8s
serviceMonitorNamespaceSelector: {}
serviceMonitorSelector: {}
version: v2.11.0
additionalScrapeConfigs:
  name: ingress-nginx-additional-configs
  key: prometheus-additional.yaml
</code>

Re‑apply the Prometheus configuration and verify the targets are up in the UI.

Import Grafana dashboard 9614 to visualise the metrics.

ServiceMonitor‑Based Scrape

When Prometheus is deployed via the Operator, use the

ServiceMonitor

CRD. Create a ServiceMonitor that selects the ingress‑nginx pods and points to the

https-metrics

port.

<code>apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: nginx-ingress
  namespace: monitoring
  labels:
    app.kubernetes.io/component: controller
spec:
  jobLabel: app.kubernetes.io/component
  endpoints:
  - port: https-metrics
    interval: 10s
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
  namespaceSelector:
    matchNames:
    - ingress-nginx
</code>

Apply the resource and verify the new target appears in Prometheus.

References

https://www.amd5.cn/atang_4421.html

https://www.cnblogs.com/lvcisco/p/12574532.html

https://prometheus.io/docs/prometheus/latest/configuration/configuration