CephFS snapshots through CSI

Advanced⏱ ~17 minkubectlceph

What you'll learn

Create and restore CephFS CSI snapshots
Understand the clone mechanism and its cost
Manage snapshot retention
Recognise the performance implications

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

CephFS snapshots and clones behave differently from RBD’s, particularly in how a clone is materialised.

Creating a snapshot

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
  name: cephfs-snapclass
driver: cephfs.csi.ceph.com
parameters:
  clusterID: <fsid>
  csi.storage.k8s.io/snapshotter-secret-name: csi-cephfs-secret
  csi.storage.k8s.io/snapshotter-secret-namespace: ceph-csi
deletionPolicy: Delete

SUBVOL=subvol
SNAP=backup-20260818
ceph fs subvolume snapshot ls cephfs ${SUBVOL} csi
ceph fs subvolume snapshot info cephfs ${SUBVOL} ${SNAP} csi

The clone mechanism

RBD restore:    a copy-on-write clone; instant, shares data with the parent
CephFS restore: a full copy of the subvolume's data

SUBVOL=subvol
SNAP=backup-20260818
NEWSUBVOL=newsubvol
ceph fs subvolume snapshot clone cephfs ${SUBVOL} ${SNAP} ${NEWSUBVOL} --group_name csi
ceph fs clone status cephfs ${NEWSUBVOL} --group_name csi

{
  "status": {
    "state": "in-progress",
    "source": { "volume": "cephfs", "subvolume": "csi-vol-...", "snapshot": "..." }
  }
}

A CephFS clone copies the data, so restoring a large subvolume takes time proportional to its size — unlike RBD, where the clone is instant.

# the PVC stays Pending until the clone completes
# the PVC created from the snapshot:
RESTORED_PVC=pgdata-restored

kubectl get pvc "$RESTORED_PVC" -w

Consistency

A CephFS snapshot captures the directory tree at a point in time.
It does not quiesce applications writing into it.
Files being written are captured mid-write.

Level	Achieved by
Crash-consistent	a plain snapshot
Application-consistent	quiesce the application first

For a shared volume with several pods writing, quiescing means coordinating all of them, which is often impractical — so CephFS snapshots are usually crash-consistent in practice.

Retention

# subvolume name from `ceph fs subvolume ls cephfs csi`:
SUBVOL=csi-vol-8f1d3c9a-4b2e-11f0-9c7a-0242ac110002

ceph fs subvolume snapshot ls cephfs "$SUBVOL" csi | \
  python3 -c 'import sys,json; [print(s["name"]) for s in json.load(sys.stdin)]'

# snapshots consume space as the subvolume changes
SUBVOL=csi-vol-8f1d3c9a-4b2e-11f0-9c7a-0242ac110002

ceph df detail
ceph fs subvolume info cephfs "$SUBVOL" csi | grep bytes

Each snapshot retains the data as it was, so an actively changing subvolume with many snapshots consumes substantially more than its current size.

Performance implications

Aspect	Effect
Snapshot creation	fast; metadata operation
Snapshot deletion	can be slow; data must be reclaimed
Clone creation	proportional to data size
Many snapshots on one subvolume	metadata overhead, slower operations
Snapshots on a busy subvolume	more retained data

# snapshot deletion is asynchronous
SUBVOL=subvol
SNAP=backup-20260818
ceph fs subvolume snapshot rm cephfs ${SUBVOL} ${SNAP} csi
ceph -s | grep -i purge

Quiz

Knowledge check · 4 questions

Q1. Why is restoring from a CephFS snapshot not instant like RBD?

CephFS snapshots are stored outside the data pool, so a restore has to read the whole snapshot back over the client network before the volume exists A CephFS clone copies the subvolume's data, because copy-on-write across an arbitrary directory tree would require per-inode indirection in the MDS The MDS must replay the snapshot's metadata journal before the subvolume can be mounted, which is the slow part CephFS snapshots are compressed when written, so a restore decompresses the whole subvolume before the PVC binds
Q2. A PVC restoring from a large CephFS snapshot can legitimately stay Pending for a long time.

True False
Q3. Plan a restore from a CephFS snapshot.

A 5 TB CephFS subvolume needs restoring from a snapshot after an application error. The team expects it to be instant based on their RBD experience.
Q4. Why does an actively changing subvolume with many snapshots consume more than its current size?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Set the expectation that a CephFS restore copies the data — a 5 TB subvolume moves 5 TB before the PVC binds, unlike an RBD clone. Check ceph fs clone status rather than treating a long-Pending PVC as a failure, and consider mounting the snapshot to copy specific files instead.

Cross-course references

Kubernetes: restore times vary enormously by storage backend and must be measured
Linux: filesystem-level snapshots and block-level ones have different clone semantics