How to Quickly Deploy Ceph NFS Service for Easy Client Mounting

1. Preparation

Check that the NFS manager module is loaded and enable it if necessary.

# ceph mgr module ls | grep nfs
# ceph mgr module enable nfs

Verify that an nfs‑ganesha cluster exists.

# ceph nfs cluster ls

2. Create NFS Service

Use the automatic creation command to create an NFS cluster. The numeric ID after create is the cluster ID and can be any value you choose; the quoted host list specifies the nodes that will run the NFS daemons.

# ceph nfs cluster create 1 "ceph01 ceph02"
NFS Cluster Created Successfully
# ceph nfs cluster ls
1

The 1 after create is the cluster ID, not the number of clusters.

The hosts inside the quotes are the nodes where the NFS processes will start.

2.1 Ceph NFS Architecture

Ceph’s NFS service relies on the nfs‑ganesha component, while CephFS is provided by the MDS daemon. The NFS‑ganesha daemon translates NFS client requests into CephFS operations, so clients see an NFS mount but the underlying storage remains CephFS.

Data can also be stored in RGW, though support is limited.

3. NFS Export

First, ensure a CephFS filesystem exists (skip if already created).

# ceph fs volume create cephfs
# ceph osd pool ls

Then create an NFS export that maps a CephFS path to an NFS pseudo‑path.

# ceph nfs export create cephfs --cluster-id 1 --pseudo-path /cephfs --fsname cephfs --path=/
{
  "bind": "/cephfs",
  "fs": "cephfs",
  "path": "/",
  "cluster": "1",
  "mode": "RW"
}

--cluster-id

matches the ID created earlier. --fsname specifies which CephFS to use. --path is the CephFS directory to export.

3.1 Client Mount

Mount the exported NFS share on a client machine.

# mount -t nfs 172.16.1.20:/cephfs /mnt
# df | grep mnt
172.16.1.20:/cephfs  59736064        0  59736064   0% /mnt

Replace the IP with your own NFS service address.

3.2 Verification

Authorize a CephFS client, mount the CephFS directly, write a file, and confirm the file appears via NFS.

# ceph fs authorize cephfs client.cephfs / rw -o ceph.client.cephfs.keyring
# cat ceph.client.cephfs.keyring
AQBTNHFmDhxSABAAqB69R7Y3Rb89LA06R0pfmw==
# mount -t ceph 172.16.1.20:6789:/ /mnt -o name=cephfs,secretfile=./ceph.client.cephfs.keyring
# echo hello > cephfs
# ls /mnt
cephfs
# cat /mnt/cephfs
hello

The data written through CephFS is visible via the NFS mount, confirming the export works.

4. High‑Availability NFS‑Ganesha

Ceph can provide HA for NFS‑ganesha using HAProxy and Keepalived. Example service definition (YAML‑style) shows placement on two hosts, one NFS daemon per host, custom frontend port (20490) to avoid conflict, and a virtual IP for failover.

service_type: ingress
service_id: nfs.1
placement:
  hosts:
    - ceph01
    - ceph02
  count_per_host: 1
spec:
  backend_service: nfs.1
  frontend_port: 20490
  monitor_port: 9000
  virtual_ip: 172.16.1.100/24

Similar HA setups can be applied to RGW services.