How to Use Ceph RBD as Persistent Storage in Kubernetes: Step‑by‑Step Guide

If you need to use RBD as a backend storage for Kubernetes, first install ceph-common.

1. Create RBD pool and image in Ceph

# ceph osd pool create pool01
# ceph osd pool application enable pool01 rbd
# rbd pool init pool01
# rbd create pool01/test --size 10G --image-format 2 --image-feature layering
# rbd info pool01/test

2. Write Kubernetes Deployment YAML

apiVersion: apps/v1
kind: Deployment
metadata:
  name: rbd
spec:
  replicas: 1
  selector:
    matchLabels:
      app: rbd
  template:
    metadata:
      labels:
        app: rbd
    spec:
      volumes:
      - name: test
        rbd:
          fsType: xfs
          keyring: /root/admin.keyring
          monitors:
          - 192.168.200.230:6789
          pool: pool01
          image: test
          user: admin
          readOnly: false
      containers:
      - name: test
        image: nginx
        volumeMounts:
        - mountPath: /usr/share/nginx/html
          name: test

After applying, the pod uses RBD as storage. If the pod stays in ContainerCreating, ensure ceph-common is installed and the keyring or ceph.conf is available on the node.

2.1 Check mount inside container

# kubectl exec -it rbd-5db4759c-nj2b4 -- bash
# df -hT | grep /dev/rbd0
/dev/rbd0      xfs       10G  105M  9.9G   2% /usr/share/nginx/html

The device is formatted as XFS and mounted at /usr/share/nginx/html.

2.2 Modify container content

# kubectl exec -it rbd-5db4759c-nj2b4 -- bash
# cd /usr/share/nginx/html
# echo 123 > index.html
# chmod 644 index.html

Accessing the pod’s IP returns 123, confirming the data is stored on RBD.

3. Use PersistentVolume (PV) with RBD

# vim pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: myclaim
spec:
  accessModes:
  - ReadWriteOnce
  volumeMode: Block
  resources:
    requests:
      storage: 8Gi

After creating a matching PV, the claim binds to the volume.

# vim pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: rbdpv
spec:
  capacity:
    storage: 8Gi
  volumeMode: Block
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  mountOptions:
  - hard
  - nfsvers=4.1
  rbd:
    fsType: xfs
    image: test
    keyring: /etc/ceph/ceph.client.admin.keyring
    monitors:
    - 172.16.1.33
    pool: rbd
    readOnly: false
    user: admin

Bind the PVC to the PV and use it in a pod.

# vim pod-pvc.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pvc-pod
spec:
  volumes:
  - name: rbd
    persistentVolumeClaim:
      claimName: myclaim
      readOnly: false
  containers:
  - name: pvc-pod
    image: nginx
    volumeDevices:
    - devicePath: /dev/rbd0
      name: rbd

4. Dynamic provisioning with StorageClass

Clone the Ceph CSI repository and adjust configuration files (namespace, extra‑create‑metadata, etc.). Create the CSI namespace: # kubectl create ns csi Modify csi-rbdplugin-provisioner.yaml and csi-config-map.yaml to set the correct cluster ID, monitors, and secret names.

4.1 Create StorageClass

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: csi-rbd-sc
provisioner: rbd.csi.ceph.com
parameters:
  clusterID: c1f213ae-2de3-11ef-ae15-00163e179ce3
  pool: rbd
  imageFeatures: "layering"
  csi.storage.k8s.io/provisioner-secret-name: csi-secret
  csi.storage.k8s.io/provisioner-secret-namespace: csi
  csi.storage.k8s.io/controller-expand-secret-name: csi-secret
  csi.storage.k8s.io/controller-expand-secret-namespace: csi
  csi.storage.k8s.io/node-stage-secret-name: csi-secret
  csi.storage.k8s.io/node-stage-secret-namespace: csi
  csi.storage.k8s.io/fstype: ext4
reclaimPolicy: Retain
allowVolumeExpansion: true
mountOptions:
- discard

4.2 Create PVC using the StorageClass

# vim sc-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: sc-pvc
spec:
  accessModes:
  - ReadWriteOnce
  volumeMode: Block
  storageClassName: "csi-rbd-sc"
  resources:
    requests:
      storage: 15Gi

Apply the PVC; the CSI driver automatically creates the PV.

4.3 Set StorageClass as default

# kubectl edit sc csi-rbd-sc
# add annotation:
#   storageclass.kubernetes.io/is-default-class: "true"

Now PVCs without an explicit storageClassName will use this class.