5 Native Commands to Diagnose Kubernetes Network Failures in Minutes

Why Kubernetes Network Troubleshooting Is Harder Than Log Analysis

Network issues in Kubernetes involve multiple encapsulation layers: pod IP → host IP → Service ClusterIP → target pod IP. A break at any layer leaves no obvious error message, unlike application logs that show explicit errors.

Command 1: Test Connectivity from Inside a Pod with curl -v

Enter the pod and send a request directly to the target Service's ClusterIP or fully qualified domain name:

# Enter a business pod
kubectl exec -it pod-name -- /bin/sh

# Test connectivity to the target Service
curl -v http://my-service.default.svc.cluster.local:8080

Key insight: The -v flag reveals detailed handshake information. If the output stalls at TCP_NODELAY , the network layer works but the application process isn't listening. If it shows Could not resolve host , DNS resolution is the problem.

Command 2: Spin Up a Temporary "Scout Pod" with kubectl run

Business pods are often minimal (distroless) and lack curl. Instead of rebuilding images, launch a throwaway pod with full networking tools:

# Start a temporary pod with alpine/curl, auto-removed on exit
kubectl run debug-pod --image=alpine/curl --rm -it --restart=Never -- /bin/sh

# Inside the pod, test the target
curl my-service:8080
nslookup my-service

The --rm flag ensures the pod disappears after exit, keeping the cluster clean. This effectively gives you a standalone machine inside the cluster for network probing.

Command 3: Verify DNS Resolution and Pod DNS Policy

Many connectivity failures stem from DNS misconfiguration. Check the pod's DNS policy and test resolution:

# Inspect the pod's dnsPolicy
kubectl get pod pod-name -o yaml | grep -A 5 dnsPolicy

# Test DNS resolution from inside the pod
kubectl exec -it pod-name -- nslookup my-service.default.svc.cluster.local

If the resolved IP doesn't match the Service's ClusterIP, or resolution times out, investigate CoreDNS or the kube-dns Service health before chasing network plugins.

Command 4: Check Service Endpoints with kubectl get endpoints

A common cause of connection refused is a Service selector that doesn't match any pod labels, leaving the Endpoints list empty.

# List endpoints for the Service
kubectl get endpoints my-service

# If ENDPOINTS is empty, inspect the Service selector
kubectl get svc my-service -o yaml | grep selector

Experience note: Selector mismatches account for roughly 40% of network connectivity incidents. Always verify endpoints before blaming the CNI or firewall.

Command 5: Capture Packets on the Host with tcpdump

If the first four steps don't reveal the issue, the problem likely resides in the host network interface or CNI plugin. Run tcpdump on the node where the pod runs:

# Capture traffic on port 8080 on Calico interfaces
# (use eth0 or the appropriate host interface if not using Calico)
tcpdump -i cali+ port 8080 -n

Interpretation: If you see SYN packets but no SYN-ACK, traffic leaves the pod but is dropped by the CNI or host firewall. If no SYN appears, the application never sent the packet.

Golden Troubleshooting Workflow (3 Steps, ~3 Minutes)

The author's daily routine for network alerts:

# 1. Verify Service has endpoints
kubectl get ep <code><your-service-name></code>

# 2. Test connectivity from a temporary pod
kubectl run test-pod --image=alpine/curl --rm -it --restart=Never -- curl -v http://<code><your-service-fqdn></code>

# 3. If still failing, capture packets on the host node
tcpdump -i eth0 port 8080 -n

This sequence isolates 90% of Kubernetes network faults within minutes, avoiding deep dives into Calico, Flannel, or other CNI configuration files.