Subvolume snapshots and asynchronous clones

Why this matters in production

The subvolume interface is what CSI and orchestration layers use, and its clone operation behaves quite differently from an RBD clone.

Managing snapshots

ceph fs subvolume snapshot create cephfs acme snap-20260818 --group_name tenants
ceph fs subvolume snapshot ls cephfs acme --group_name tenants

ceph fs subvolume snapshot info cephfs acme snap-20260818 --group_name tenants

{
    "created_at": "2026-08-18 02:00:11.482913",
    "data_pool": "cephfs.cephfs.data",
    "has_pending_clones": "no"
}

ceph fs subvolume snapshot rm cephfs acme snap-20260818 --group_name tenants

A snapshot with pending clones cannot be removed until they complete,
which is what `has_pending_clones` reports.

Cloning

ceph fs subvolume snapshot clone cephfs acme snap-20260818 acme-recovered \
  --group_name tenants --target_group_name tenants

ceph fs clone status cephfs acme-recovered --group_name tenants

{ "status": { "state": "in-progress",
              "source": { "volume": "cephfs", "subvolume": "acme",
                          "snapshot": "snap-20260818" } } }

The clone is a full copy performed asynchronously by the manager. It is
not a copy-on-write reference like an RBD clone.

ceph fs clone cancel cephfs acme-recovered --group_name tenants

Managing clone progress

ceph config get mgr mgr/volumes/max_concurrent_clones
ceph config set mgr mgr/volumes/max_concurrent_clones 4

Clones are copies, so they consume capacity equal to the source and
generate read and write load for their duration. Concurrency is the
control that keeps a batch of clones from saturating the cluster.

# CLONE is the target name given to `ceph fs subvolume snapshot clone` above:
CLONE=acme-recovered

# what is running
ceph fs clone status cephfs "$CLONE" --group_name tenants
ceph -s | grep -i -E 'client io|recovery'

# capacity before starting
ceph df detail | head
ceph fs subvolume info cephfs acme --group_name tenants | python3 -c '
import sys,json; print("source used:", json.load(sys.stdin).get("bytes_used"))'

Using clones for recovery

The recovery shape CephFS offers:
  clone the snapshot into a new subvolume
  wait for the clone to complete
  take what is needed, or repoint the consumer at the clone

ceph fs subvolume getpath cephfs acme-recovered --group_name tenants

# MON is a monitor address from `ceph mon dump`; substitute your own:
MON=192.0.2.11:6789

# the clone's path, including its UUID, from the getpath call above
SUBVOL_PATH=$(ceph fs subvolume getpath cephfs acme-recovered --group_name tenants)

# mount the clone's path and copy what is needed
mount -t ceph "$MON:$SUBVOL_PATH" /mnt/recovered \
  -o name=admin,secretfile=/run/ceph.key

For a single file, reading directly from the source subvolume's .snap
directory is far faster than cloning — no copy is involved at all.

Quiz

Production discipline

Read from the source subvolume’s .snap path to recover individual files — a subvolume clone copies every byte and takes capacity and time proportional to the whole subvolume. Bound mgr/volumes/max_concurrent_clones before any bulk restore.

Cross-course references

• Kubernetes: restoring from a CSI snapshot provisions a full new volume the same way
• Linux: prefer reading from a snapshot over materialising a copy of it