Master Kubernetes Troubleshooting: 10 Essential Steps to Diagnose Pods, Networks, and DNS

1. Pod Startup Issues

Pods are the smallest scheduling unit in Kubernetes; containers inside a pod share the pod's network, storage, and resources. Common causes of pod failures include:

Resource exhaustion: too many pods on a node overload the node.

Memory/CPU limits exceeded: application memory leaks cause the pod to be killed. Use load testing and set resource limits.

Network problems: e.g., mis‑configured Calico plugin.

Storage problems: shared storage or volume mount failures.

Code errors: application crashes on start.

Configuration errors: incorrect Deployment or StatefulSet manifests.

Monitoring: use observability tools to spot these issues.

2. Inspect Cluster State

Start by checking node health with kubectl get nodes. Ensure core components such as etcd, kubelet, and kube-proxy are running.

3. Trace Event Logs

View cluster events using kubectl get events to identify component‑level errors.

4. Focus on Pod Status

List all pods across namespaces: kubectl get pods --all-namespaces. For problematic pods, run kubectl describe pod <pod-name> to get detailed information.

5. Check Network Connectivity

Verify service, pod, and node communication. Use kubectl get services and kubectl describe service <svc-name>. Review network policies and firewall rules.

6. Review Storage Configuration

If your workloads use Persistent Volumes or StorageClasses, check their status with kubectl get pv, kubectl get pvc, and kubectl get storageclass.

7. Examine Container Logs

Fetch logs with kubectl logs <pod-name>. For multi‑container pods, specify the container: kubectl logs <pod-name> -c <container-name>.

8. Kubernetes Cluster Network Communication

The cluster relies on a CNI plugin (e.g., Calico, Flannel). Below is a diagram of typical network flow.

Key points:

Calico provides IP address allocation and network policies with performance comparable to Flannel.

Flannel only supports IP address allocation.

Canel (a hybrid) combines features of Calico and Flannel.

Network communication types in a cluster:

Communication between containers within the same pod.

Pod‑to‑Pod communication.

Pod‑to‑Service communication.

Service communication with external clients.

9. Service DNS Verification

Test DNS resolution from a pod in the same namespace:

u@pod$ nslookup hostnames
Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local
Name: hostnames
Address 1: 10.0.1.175 hostnames.default.svc.cluster.local

If it fails, try a fully qualified name:

u@pod$ nslookup hostnames.default.svc.cluster.local

Check /etc/resolv.conf to ensure the DNS service IP and search domains are correct:

nameserver 10.0.0.10
search default.svc.cluster.local svc.cluster.local cluster.local example.com
options ndots:5

10. Summary

The exact troubleshooting steps depend on your cluster configuration and the symptoms observed. By following the above checklist—examining pod status, node health, network, storage, logs, and DNS—you can more confidently diagnose and resolve Kubernetes issues, keeping your applications stable and reliable.