File-level and object-level backup paths

Intermediate⏱ ~18 minrsyncrcloneceph

What you'll learn

Compare backup at the file, object, and block layers
Back up CephFS and RGW contents
Recognise where file-level backup is the correct choice
Handle the scale problems it creates

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

Backing up at the file layer produces a copy anything can read, which is often worth more than efficiency.

The three layers

Layer	Produces	Restores
Block (`rbd export`)	an image file	the whole image
File (`rsync` from a mount)	a file tree	any subset, anywhere
Object (`rclone`, `s3 sync`)	objects in a bucket	any subset

The file and object layers give granular restore and portability. The
block layer gives efficiency and crash consistency.

# file layer, from a CephFS mount
rsync -aHAX --numeric-ids --delete \
  /mnt/cephfs/tenants/acme/ backup-host:/backups/acme/

# object layer, RGW to an external target
rclone sync ceph-rgw:acme-bucket external-s3:acme-backup \
  --transfers 16 --checksum

Where file-level is the correct choice

Situation	Why
Restores are usually of single files	granular restore is the point
The backup target is not Ceph	portability matters
The data is already a file tree	no translation needed
Long retention with deduplication	file-level tools handle it
Restore must be possible without Ceph	the copy is readable anywhere

The last is the one that decides it for many organisations: an image
file from rbd export needs Ceph or a compatible tool to open. A file
tree needs nothing.

The scale problems

File-level backup walks the tree. On a filesystem with tens of millions
of files, the walk itself becomes the bottleneck.

ceph fs status
ceph tell mds.0 perf dump 2>/dev/null | python3 -c '
import sys,json
d = json.load(sys.stdin).get("mds", {})
print("inodes:", d.get("inodes"), " caps:", d.get("caps"))'

Problem	Mitigation
Metadata walk dominates	use snapshots and diff, not full walks
MDS load from the walk	run backups against a snapshot mount
Backup window exceeded	parallelise by subtree
Changed-file detection is expensive	use `rsync` with a file list from a snapshot diff

# back up from a snapshot rather than the live tree
# The subvolume's UUID directory, from
# `ceph fs subvolume getpath cephfs acme --group_name tenants`:
SUBVOL_UUID=4d2b8c1e-9f0a-4c7d-b2e1-8a3f5c6d7e90

ceph fs subvolume snapshot create cephfs acme snap-nightly --group_name tenants
# the snapshot is visible under the subvolume's .snap directory
rsync -aHAX "/mnt/cephfs/volumes/tenants/acme/$SUBVOL_UUID/.snap/snap-nightly/" \
  backup-host:/backups/acme/

Backing up from a snapshot gives a consistent view and removes the race
between the walk and ongoing writes.

Verifying

# counts and sizes should agree
find /mnt/cephfs/tenants/acme -type f | wc -l
ssh backup-host 'find /backups/acme -type f | wc -l'

rclone check ceph-rgw:acme-bucket external-s3:acme-backup --one-way

Quiz

Knowledge check · 4 questions

Q1. Why back up CephFS from a snapshot rather than the live tree?

A walk of a live tree races ongoing writes, so the copy reflects no single point in time — some files from the start of the walk, some from the end Reading from a snapshot is faster, because the MDS serves snapshot metadata from its cache rather than walking the live inode tree for every directory entry Snapshot data compresses better on the backup target, since the objects are already immutable by the time the walk reads them The live tree cannot be read while a backup runs, because the MDS holds an exclusive capability for the duration of the walk
Q2. Block-level backup is always preferable to file-level backup because it is more efficient.

True False
Q3. Choose a backup layer.

A CephFS filesystem holds a departmental file share. Restores are almost always of a single accidentally-deleted file, and the backup target is a generic NAS.
Q4. What becomes the bottleneck in file-level backup at scale?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Create a snapshot and read from .snap rather than walking the live tree — a live walk races ongoing writes and produces a copy that reflects no single point in time. Choose file-level backup where restores are granular or the target is not Ceph.

Cross-course references

Kubernetes: volume snapshots before backup exist for the same consistency reason
Linux: a filesystem walk during writes captures no coherent moment