Skip to main content
RunBook Academy

CephXXXVII · RBD SnapshotsRBD Snapshots

What snapshots protect against, and what they do not

Intermediate⏱ ~16 minrbd

What you'll learn

  • Enumerate the failures snapshots do and do not cover
  • Position snapshots within a layered protection strategy
  • Configure off-cluster protection appropriately
  • Communicate the distinction accurately

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Not yet marked complete on this device.

Why this matters in production

Snapshots are frequently the only protection a volume has, and they are described in planning documents as backups. They are effective against a real and common class of failure and useless against another, and the documents rarely distinguish.

The coverage table

FailureSnapshot covers it?
User deleted files inside the guestyes
Bad application upgradeyes
Guest filesystem corruptionyes
Ransomware encrypting guest filesyes, if the snapshot predates it
Accidental rbd rm of the imageno — snapshots go with it
Pool deletionno
Cluster loss or site failureno
Correlated OSD failure destroying the datano
Malicious admin with cluster accessno

The pattern: snapshots cover failures above the storage layer and nothing at or below it.

The layered strategy

graph TD
    A[Guest-level mistakes] --> B[Snapshots same pool, instant recovery]
    C[Image or pool loss] --> D[RBD mirroring second cluster]
    E[Site loss] --> F[Off-site export independent media]
    G[Malicious deletion] --> H[Immutable off-site copies]

Each layer covers what the one above cannot:

# layer 1: snapshots
rbd snap create rbd-vms/vm-disk-01@hourly-$(date -u +%H)

# layer 2: mirroring to another cluster
rbd mirror pool enable rbd-vms image
rbd mirror image enable rbd-vms/vm-disk-01 snapshot

# layer 3: export to independent storage
rbd export-diff --from-snap daily-prev \
    rbd-vms/vm-disk-01@daily-now - | \
    ssh backup-host 'cat > /backup/vm-disk-01-incr.diff'

Mirroring is not a backup either

RBD mirroring replicates changes to a second cluster, including deletions. A rm -rf inside the guest replicates faithfully. Mirroring protects against cluster and site loss; it does not protect against mistakes, because it propagates them.

Only retained, independent copies — exports with retention, or snapshots on the mirror side kept independently — protect against mistakes that replicate.

Communicating it

State coverage in terms of failures, not mechanisms:

Guest-level mistakes: recoverable within 7 days from hourly snapshots. Cluster loss: recoverable to within 15 minutes from the mirror cluster. Site loss: recoverable to within 24 hours from off-site exports.

That is a statement someone can plan against. “We take snapshots” is not.

Quiz

Knowledge check · 4 questions

  1. Q1. A user runs `rm -rf` inside a VM whose volume is mirrored to a second cluster. What happens on the mirror?

  2. Q2. Snapshots protect against accidental deletion of the image they belong to.

  3. Q3. Assess a data-protection strategy against stated requirements.

    A team documents their protection as "hourly RBD snapshots retained for 7 days, mirrored to a DR cluster". Management asks what happens if a script accidentally deletes 40 production images.

  4. Q4. Why are replication and retention orthogonal properties?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Document protection coverage as a list of failures with recovery points, not as a list of mechanisms — the mechanism list is what allows a gap this large to go unnoticed. Ensure at least one layer has an existence independent of the image itself, since image deletion is the failure most strategies quietly fail to cover.

Cross-course references

  • Kubernetes: etcd replication versus etcd backups is exactly this distinction
  • Linux: RAID protects the medium, backups protect the content — the same orthogonality