Step-by-Step Guide to Deploying a Ceph Cluster with cephadm

Cephadm Introduction

Since Red Hat Ceph 5, cephadm replaces ceph‑ansible for the full lifecycle of a Ceph cluster, handling deployment, management, and monitoring.

The bootstrap process creates a small storage cluster on a single bootstrap node, including a Ceph Monitor, a Ceph Manager, and all required dependencies.

Cephadm pulls container images from a registry and runs them on each Ceph node; the containers are essential because the deployed Ceph services run inside them.

Communication with cluster nodes is performed via SSH, allowing cephadm to add hosts, storage, and monitor them.

The bootstrap node must have cephadm, podman or docker, python3, and chrony installed, reducing deployment complexity.

Prerequisites

1. Install Python 3: yum -y install python3 2. Install Docker (or Podman):

# Install Docker CE from Alibaba Cloud mirror
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
yum -y install docker-ce
systemctl enable docker --now
# Configure image mirror
mkdir -p /etc/docker
echo '{"registry-mirrors": ["https://bp1bh1ga.mirror.aliyuncs.com"]}' > /etc/docker/daemon.json
systemctl daemon-reload
systemctl restart docker

3. Install a time‑synchronization service (chrony or NTP).

Node Preparation

Four Rocky Linux 8.6 nodes are used (node1‑node4). Their roles and disks are:

node1: mon, mgr, server, admin node – /dev/vdb, /dev/vdc, /dev/vdd

node2: mon, mgr – /dev/vdb, /dev/vdc, /dev/vdd

node3: mon, mgr – /dev/vdb, /dev/vdc, /dev/vdd

node4: client, admin node – no disks assigned

Update /etc/hosts on each node:

172.24.1.6 node1
172.24.1.7 node2
172.24.1.8 node3
172.24.1.9 node4

Enable password‑less SSH from node1 to the other nodes:

# ssh-keygen
# ssh-copy-id root@node2
# ssh-copy-id root@node3
# ssh-copy-id root@node4

Installing cephadm on node1

# yum -y install epel-release
# yum search release-ceph
# yum -y install centos-release-ceph-pacific.noarch
# yum -y install cephadm
# yum -y install ceph-common

Deploying the Ceph Cluster

Bootstrap the cluster and install the dashboard:

# cephadm bootstrap --mon-ip 172.24.1.6 \
    --allow-fqdn-hostname \
    --initial-dashboard-user admin \
    --initial-dashboard-password redhat \
    --dashboard-password-noupdate

The command verifies Docker/Podman, LVM, and chrony, pulls the Ceph container image, creates monitors, managers, and enables the dashboard.

Copy the cluster public key to the other nodes:

# ssh-copy-id -f -i /etc/ceph/ceph.pub root@node2
# ssh-copy-id -f -i /etc/ceph/ceph.pub root@node3
# ssh-copy-id -f -i /etc/ceph/ceph.pub root@node4

Add the remaining hosts to the orchestrator:

# ceph orch host add node2 172.24.1.7
# ceph orch host add node3 172.24.1.8
# ceph orch host add node4 172.24.1.9

Label node1 and node4 as administrators and copy configuration files to node4:

# ceph orch host label add node1 _admin
# ceph orch host label add node4 _admin
# scp /etc/ceph/{*.conf,*.keyring} root@node4:/etc/ceph

Deploy monitors, managers, and OSDs:

# ceph orch apply mon "node1,node2,node3"
# ceph orch apply mgr --placement="node1,node2,node3"
# for i in node1 node2 node3; do
    for j in vdb vdc vdd; do
        ceph orch daemon add osd $i:/dev/$j
    done
  done

Verify the cluster status:

# ceph -s

Managing the Cluster from node4

Install the Ceph repository and common package on node4, then check status:

# yum -y install centos-release-ceph-pacific.noarch
# yum -y install ceph-common
# ceph -s

The output shows HEALTH_OK, three monitors, one manager, and nine OSDs.

For further details, refer to the original blog post.