How to Deploy Ceph NFS Service with High Availability – Step-by-Step Guide

Ceph NFS Service Deployment

Ceph can provide NFS services in addition to CephFS, offering a simpler client mount experience.

1. Preparation

1.1 Check NFS module

# ceph mgr module ls | grep nfs
"nfs",

If the module is not enabled, run:

# ceph mgr module enable nfs

1.2 Verify nfs‑ganesha cluster

# ceph nfs cluster ls

2. Create NFS Service

Use the automatic creation command:

# ceph nfs cluster create 1 "ceph01 ceph02"
NFS Cluster Created Successfully
# ceph nfs cluster ls
1

The number after create is the cluster ID (can be changed).

The quoted hosts (ceph01, ceph02) indicate on which nodes the processes run.

2.1 NFS Architecture in Ceph

Ceph’s NFS service uses the nfs‑ganesha component, while CephFS is provided by the MDS component. nfs‑ganesha acts as a translator, converting NFS requests into CephFS operations.

Data flow: Client request → nfs‑ganesha → MDS . Storage can also be directed to RGW, though support is limited.

3. NFS Export (Configure Backend Storage)

3.1 Create CephFS (if not existing)

# ceph fs volume create cephfs
# ceph osd pool ls
device_health_metrics
.nfs
cephfs.cephfs.meta
cephfs.cephfs.data

3.2 Export

# ceph nfs export create cephfs --cluster-id 1 --pseudo-path /cephfs --fsname cephfs --path=/
{
  "bind": "/cephfs",
  "fs": "cephfs",
  "path": "/",
  "cluster": "1",
  "mode": "RW"
}

The export creates an NFS share from the CephFS named cephfs at path /cephfs. --fsname selects the CephFS instance; --path specifies the CephFS directory.

3.3 Client Mount Attempt

# mount -t nfs 172.16.1.20:/cephfs /mnt
# df | grep mnt
172.16.1.20:/cephfs  59736064        0  59736064   0% /mnt

Replace the IP with your NFS service address. Mounting via NFS is simpler than direct CephFS mounts because it avoids creating CephFS users and authorizations.

3.4 Verification

Authorize a CephFS client and mount the CephFS directly to verify data visibility.

# ceph fs authorize cephfs client.cephfs / rw -o ceph.client.cephfs.keyring
# cat ceph.client.cephfs.keyring
AQBTNHFmDhxSABAAqB69R7Y3Rb89LA06R0pfmw==
# mount -t ceph 172.16.1.20:6789:/ /mnt -o name=cephfs,secretfile=./ceph.client.cephfs.keyring
# echo hello > cephfs

List the file via the NFS mount:

# ls
cephfs
# cat cephfs
hello

The data written through CephFS is visible via the NFS export.

4. High‑Availability nfs‑ganesha

Ceph can make nfs‑ganesha highly available using HAProxy and keepalived. Example service definition:

service_type: ingress
service_id: nfs.1  # NFS ID
placement:
  hosts:
    - ceph01
    - ceph02
  count_per_host: 1  # Number of NFS processes per node
spec:
  backend_service: nfs.1
  frontend_port: 20490  # Changed from default 2049 because 2049 was already in use
  monitor_port: 9000
  virtual_ip: 172.16.1.100/24

Both NFS and RGW can be deployed with high availability using this approach.

Reference: Original article