CephFS StorageClass configuration

Advanced⏱ ~17 minkubectlceph

What you'll learn

Configure a CephFS StorageClass fully
Choose between the kernel and FUSE mounters
Configure per-tenant isolation
Tune mount options where warranted

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

The CephFS StorageClass has fewer parameters than RBD’s, and the ones it has affect isolation and performance directly.

A complete StorageClass

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ceph-fs
provisioner: cephfs.csi.ceph.com
parameters:
  clusterID: <fsid>
  fsName: cephfs
  pool: cephfs_data
  subvolumeGroup: csi
  mounter: kernel
  kernelMountOptions: "readdir_max_entries=1024"
  csi.storage.k8s.io/provisioner-secret-name: csi-cephfs-secret
  csi.storage.k8s.io/provisioner-secret-namespace: ceph-csi
  csi.storage.k8s.io/controller-expand-secret-name: csi-cephfs-secret
  csi.storage.k8s.io/controller-expand-secret-namespace: ceph-csi
  csi.storage.k8s.io/node-stage-secret-name: csi-cephfs-secret
  csi.storage.k8s.io/node-stage-secret-namespace: ceph-csi
reclaimPolicy: Delete
allowVolumeExpansion: true

The mounter choice

Mounter	Client	Characteristics
`kernel`	kernel CephFS driver	faster; feature support depends on kernel
`fuse`	ceph-fuse	slower; all features; a fault kills one process

mounter: kernel
kernelMountOptions: "..."
# or
mounter: fuse
fuseMountOptions: "..."

The kernel mounter is the default and correct choice unless a specific feature requires FUSE — most commonly quota enforcement behaviour or snapshot support on older kernels.

kubectl get nodes -o jsonpath='{range .items[*]}{.status.nodeInfo.kernelVersion}{"\n"}{end}' | sort -u

Per-tenant isolation

# a class per tenant, each with its own group and credentials
parameters:
  fsName: cephfs
  subvolumeGroup: team-a
  csi.storage.k8s.io/provisioner-secret-name: csi-cephfs-team-a

ceph fs subvolumegroup create cephfs team-a --size 10995116277760
ceph fs authorize cephfs client.team-a /volumes/team-a rw

Group plus scoped credentials gives each tenant a bounded, isolated area within one filesystem.

Mount options worth considering

Option	Effect
`readdir_max_entries`	larger directory reads; helps large directories
`rasize`	readahead size
`wsize`, `rsize`	I/O sizes
`noatime`	avoids metadata writes on read
`_netdev`	ordering during boot

kernelMountOptions: "noatime,readdir_max_entries=1024"

noatime is worth setting on almost any CephFS volume: access-time updates generate metadata writes for every read, which is pure MDS load for information almost nothing uses.

Verifying

# POD is a pod consuming the PVC, from `kubectl get pods`; substitute your own:
POD=team-a-web-5f9c7d8b6c-2xk9p

kubectl exec "$POD" -- mount | grep ceph
kubectl exec "$POD" -- df -h /data
ceph fs subvolume ls cephfs team-a

Quiz

Knowledge check · 4 questions

Q1. Why does `noatime` matter more on CephFS than on a local filesystem?

Each access-time update is a metadata write to the MDS and metadata pool, so a read-heavy workload generates substantial unnecessary metadata load CephFS does not record atime at all, so the option only stops the client attempting an update the MDS will reject It reduces traffic to the OSDs, because atime updates are written into the data pool alongside the file's contents rather than into the metadata pool It is required for shared mounts, since concurrent atime updates from several clients would conflict at the MDS
Q2. A distribution kernel frozen for its support lifetime can lag the Ceph cluster's CephFS features by years.

True False
Q3. Configure CephFS for multiple tenants.

A cluster will provide CephFS to three teams. Currently one StorageClass with the default subvolume group serves everyone.
Q4. When is the FUSE mounter the right choice for CephFS?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Set noatime in the kernel mount options on every CephFS volume — the access-time updates are metadata writes to the MDS for information almost nothing reads. Give each tenant its own subvolume group with a quota and path-scoped credentials; both are needed for the isolation to hold.

Cross-course references

Kubernetes: per-tenant StorageClasses encode isolation policy the same way
Linux: noatime is standard practice on any filesystem where atime is not needed