Understanding Kubernetes Storage: Ephemeral, Persistent, and Config Management

Ephemeral Storage (Volume)

emptyDir

Creates an empty directory when the Pod is scheduled; the directory is removed permanently when the Pod is deleted. Typical for temporary cache or data sharing between containers (e.g., sidecar).

volumeMounts:
  - name: config-volume
    mountPath: /etc/alertmanager
  - name: storage-volume
    mountPath: /alertmanager
resources:
  requests:
    cpu: "100m"
    memory: "128Mi"
  limits:
    cpu: "200m"
    memory: "256Mi"
volumes:
  - name: config-volume
    configMap:
      name: alertmanager-config
      items:
        - key: alertmanager.yml
          path: alertmanager.yml
        - key: email.tmpl
          path: templates/email.tmpl
  - name: storage-volume
    emptyDir: {}

hostPath

Mounts a directory from the node’s filesystem into the Pod. Data is lost if the Pod is rescheduled to a different node, so it is recommended only for testing or accessing node‑local resources.

volumes:
  - name: config
    configMap:
      name: prometheus-config
  - name: rules
    configMap:
      name: prometheus-rules
  - name: storage
    hostPath:
      path: /app/promethues
      type: DirectoryOrCreate

Persistent Storage (PV & PVC)

Core concepts

Access Modes : ReadWriteOnce (single node), ReadOnlyMany (multiple nodes read‑only), ReadWriteMany (multiple nodes read‑write, e.g., NFS, CephFS).

Reclaim Policies : Retain (PV kept after PVC deletion), Delete (PV removed automatically, common for dynamic provisioning).

Static provisioning

Used when storage resources are pre‑allocated by an administrator.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-pv
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: /data/pv
  persistentVolumeReclaimPolicy: Retain
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  resources:
    requests:
      storage: 1Gi
  accessModes:
    - ReadWriteOnce
  volumeName: my-pv
---
apiVersion: v1
kind: Pod
metadata:
  name: pv-pod
spec:
  containers:
  - name: nginx
    image: nginx:alpine
    volumeMounts:
    - name: data
      mountPath: /usr/share/nginx/html
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: my-pvc

Verification steps:

# Write a test file inside the container
kubectl exec -it pv-pod -- sh -c "echo 'hello pv-pvc' > /usr/share/nginx/html/index.html"
# Delete the Pod
kubectl delete pod pv-pod
# Recreate the Pod
kubectl apply -f pv-pvc-demo.yaml
# Confirm the file persists
kubectl exec -it pv-pod -- cat /usr/share/nginx/html/index.html

Dynamic Provisioning (StorageClass)

Provisioning workflow

User creates a PVC that references storageClassName.

The referenced StorageClass invokes its provisioner, which automatically creates a PV.

The new PV binds to the PVC; Pods can mount the PVC.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-path-storage
provisioner: rancher.io/local-path
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-dynamic-pvc
spec:
  storageClassName: local-path-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: test-pod
spec:
  containers:
  - name: test-container
    image: nginx
    volumeMounts:
    - mountPath: /usr/share/nginx/html
      name: my-storage
  volumes:
  - name: my-storage
    persistentVolumeClaim:
      claimName: my-dynamic-pvc

Config Storage (ConfigMap & Secret)

ConfigMap

Stores non‑sensitive configuration such as environment variables or configuration files.

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  LOG_LEVEL: "info"
  DB_HOST: "mysql-service"
  nginx.conf: |
    server {
      listen 80;
      server_name example.com;
      location / {
        root /usr/share/nginx/html;
        index index.html;
      }
    }
---
apiVersion: v1
kind: Pod
metadata:
  name: configmap-pod
spec:
  containers:
  - name: nginx
    image: nginx:1.23
    env:
    - name: LOG_LEVEL
      valueFrom:
        configMapKeyRef:
          name: app-config
          key: LOG_LEVEL
    - name: DB_HOST
      valueFrom:
        configMapKeyRef:
          name: app-config
          key: DB_HOST
    volumeMounts:
    - name: config-volume
      mountPath: /etc/nginx/conf.d
  volumes:
  - name: config-volume
    configMap:
      name: app-config
      items:
      - key: nginx.conf
        path: default.conf

Secret

Stores sensitive data such as passwords, API keys, or TLS certificates. Values can be provided as base64‑encoded strings ( data) or as plain text ( stringData, which the API encodes automatically).

apiVersion: v1
kind: Secret
metadata:
  name: db-secret
type: Opaque
data:
  DB_PASSWORD: bXlwYXNzd29yZA==
stringData:
  DB_USER: "admin"
---
apiVersion: v1
kind: Pod
metadata:
  name: secret-pod
spec:
  containers:
  - name: mysql
    image: mysql:8.0
    env:
    - name: MYSQL_ROOT_PASSWORD
      valueFrom:
        secretKeyRef:
          name: db-secret
          key: DB_PASSWORD
    - name: MYSQL_USER
      valueFrom:
        secretKeyRef:
          name: db-secret
          key: DB_USER
    volumeMounts:
    - name: secret-volume
      mountPath: /etc/secrets
      readOnly: true
  volumes:
  - name: secret-volume
    secret:
      secretName: db-secret

Enterprise‑grade storage plugin: Ceph

Ceph provides block storage (RBD), file storage (CephFS), and object storage (RGW). It is suitable for production workloads that require high availability.

Prerequisites:

Deployed Ceph cluster with monitors, OSDs, and MDS.

Ceph CSI driver installed (Kubernetes 1.23+ recommended).

Typical use case: Ceph RBD offers ReadWriteOnce block volumes for databases or other stateful applications.