Common Kubernetes Pod Issues and Troubleshooting Guide

1. ContainerCreating

The pod remains in ContainerCreating when underlying configuration problems prevent the container from being created, often due to Docker service issues or remote storage mount failures.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        imagePullPolicy: Always
        name: nginx
        volumeMounts:
        - mountPath: /tmp/
          name: www
      volumes:
      - name: www
        nfs:
          path: /nfs/web/date
          readOnly: true
          server: 192.168.1.21

2. ErrImagePull / ImagePullBackOff

This state occurs when the image cannot be pulled, typically because the image does not exist in the registry or the registry has been cleaned up during CI/CD pipelines.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx:1111111111  # non‑existent image
        imagePullPolicy: Always
        name: nginx
        volumeMounts:
        - mountPath: /tmp/
          name: www
      volumes:
      - name: www
        configMap:
          name: nginx-config

3. Pending

Pods stay pending when the scheduler cannot place them, which may be caused by scheduler component failures, missing ServiceAccount, incorrect node selector or tolerations, insufficient node resources, or problematic PersistentVolume claims.

1. K8s scheduler component abnormal or down
2. ServiceAccount missing or lacks permission
3. Node selector / tolerations mis‑configured
4. Insufficient node resources
5. PV volume issues

4. CrashLoopBackOff / ERROR

CrashLoopBackOff indicates that the container repeatedly fails to start. Common reasons include mis‑configured container runtime, failing health‑check probes, runtime errors in start‑up scripts, or resource limits causing OOM kills.

1. Container runtime configuration error
2. Liveness/readiness probe failure
3. Startup script errors (e.g., unreachable FTP, DNS failures)
4. Memory or CPU limits too low leading to OOM

5. UnexpectedAdmissionError

This error often appears when a node's disk is full, preventing the node from writing logs or performing other operations. Cleaning up the filesystem and deleting affected pods resolves the issue.

kubectl get pods -A | grep UnexpectedAdmissionError | awk '{print("kubectl delete pod ", $2, " -n ", $1)}' | /bin/bash

6. ConfigMap Issues

Pods may fail to start if a referenced ConfigMap does not exist or its name is misspelled. Verify the ConfigMap name and ensure it is created in the correct namespace.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        imagePullPolicy: Always
        name: nginx
        volumeMounts:
        - mountPath: /tmp/
          name: www
      volumes:
      - name: www
        configMap:
          name: nginx-config

7. NFS / GFS Storage Mount Problems

When remote NFS or GFS storage is unavailable, pods that depend on those volumes will remain in ContainerCreating or Pending . Checking pod events and verifying the storage server resolves the issue.

nfs mount error: mount failed: exit status 32 – storage does not exist
nfs mount error: exit status 1 – check mount path or permissions

gfs mount error: exit status 32 – GFS client not installed on node

gfs mount error: exit status 1 – volume not created on GFS server

8. Linux Node Issues Misidentified as K8s Problems

Node-level problems such as a hung df -h command caused by a failed NFS mount can manifest as pod errors. Use mount -l to identify problematic mounts and restart the storage service.

// Deploy NFS service
echo "/home/nfs *(rw,async,no_root_squash)" >> /etc/exports
systemctl start nfs

// On client
mkdir -p /tmp/test1/
mount -t nfs 10.0.16.16:/home/nfs /tmp/test1/

df -h | grep /tmp/test1