Kubernetes Production Pitfalls: 99% of Common Issues and Solutions

Kubernetes Core Architecture Overview

The control plane consists of kube-apiserver, scheduler, controller‑manager and etcd. Worker nodes run kubelet and kube-proxy, hosting Pods and their containers.

1. Pod‑Related Issues

1.1 Pod Stuck in Pending

Symptoms : kubectl get pods shows STATUS=Pending.

Investigation steps :

Describe the pod: kubectl describe pod <pod> Check recent events: kubectl get events --sort-by='.lastTimestamp' Inspect node resources: kubectl top nodes and kubectl get nodes -o wide Verify node schedulability:

kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.unschedulable}{"
"}{end}'

Common causes and fixes :

Insufficient resources : expand the cluster or lower the pod’s requests (e.g., cpu: 100m, memory: 128Mi) and patch the deployment.

Node selector mismatch : add the missing label to a node ( kubectl label nodes node-1 disktype=ssd) or remove the selector.

Taints : add a matching tolerations block or remove the taint from the node.

PVC not bound : ensure the referenced StorageClass exists and is provisioned.

1.2 Pod Stuck in ContainerCreating

Symptoms : STATUS=ContainerCreating.

Investigation steps :

Describe the pod for events.

Check kubelet logs: journalctl -u kubelet -f.

Check container runtime logs (Docker or containerd).

Typical reasons :

Image pull failure – create an ImagePullSecret and reference it in the pod spec.

Volume mount failure – verify StorageClass, PV status, and PVC binding.

CNI plugin problems – inspect the CNI pod logs and restart the daemonset.

1.3 CrashLoopBackOff

Inspect logs ( kubectl logs <pod> and --previous) and the exit code via

kubectl get pod -o jsonpath='{.status.containerStatuses[0].lastState.terminated.exitCode}'

. The article provides a table mapping exit codes (1, 127, 137, 143, 130) to concrete actions such as fixing application bugs, adjusting memory limits, or handling signals.

1.4 Pod Stuck in Terminating

Force delete with kubectl delete pod <pod> --grace-period=0 --force, remove finalizers via

kubectl patch pod <pod> -p '{