Enumerating the threats a backup must answer

Intermediate⏱ ~18 mincephrbd

What you'll learn

Enumerate the threats to stored data
Characterise each by reach and timing
Match a control to each threat
Identify which threats a design leaves open

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

“Backup” as a single word hides that the threats have different reach, different timing, and therefore need different controls.

The threats

Threat	Reach	Timing
Accidental deletion	one object, image, or bucket	instant
Application corruption	one dataset	gradual or instant
Operator error at scale	a pool or the cluster	instant
Ransomware	everything the credential reaches	hours to days
Bad upgrade or software defect	potentially everything	gradual
Site loss	one cluster	instant
Silent corruption	scattered objects	gradual

# what a credential can reach is what ransomware can reach
ceph auth get client.app-a

Characterising by reach

Reach determines which copies survive.
A threat that reaches the cluster is not answered by anything on it.

Threat reaches	Surviving copy must be
One image	anywhere else, including the same cluster
One pool	outside that pool
Everything one credential can touch	outside that credential’s scope
The cluster	on other infrastructure
The site	in another site
The organisation’s whole environment	offline or third-party held

# does the backup credential have write access to the backup?
ceph auth get client.backup

A backup credential with write access to the backup target means
ransomware that compromises the backup process reaches the backups.

Characterising by timing

Timing determines how far back the recovery point must reach.

Threat	Detected
Deletion	usually within hours
Ransomware	at encryption, or later
Application corruption	days or weeks later
Silent corruption	at the next read, possibly months

# scrub is what surfaces silent corruption
ceph pg dump pgs 2>/dev/null | awk '{print $1, $NF}' | head -5
ceph config get osd osd_deep_scrub_interval

Retention has to exceed the detection delay for the slowest threat that
matters, which is usually application corruption rather than the
dramatic ones.

Matching controls

Threat	Control
Accidental deletion	same-cluster snapshots with short retention
Application corruption	off-cluster copies with long retention
Operator error at scale	copies the operator credential cannot reach
Ransomware	immutable or offline copies
Site loss	mirroring or replication to another site
Silent corruption	scrubbing, plus a copy predating the corruption

# same-cluster, cheap, covers the common cases
POOL=rbd-vms
IMAGE=vm-disk-01
rbd snap create ${POOL}/${IMAGE}@$(date +%Y%m%d)
rbd snap ls ${POOL}/${IMAGE}

# Substitute your own values before running:
POOL=rbd-vms
IMAGE=vm-disk-01
SNAP=20260818

# off-cluster, covers the severe cases
rbd export "$POOL/$IMAGE@$SNAP" - | ssh backup-host 'cat > /backups/img.raw'

Finding what is open

For each threat, name the control. Any threat with no control is a
decision, and it should be recorded as one.

Quiz

Knowledge check · 4 questions

Q1. What determines how long backups must be retained?

The severity of the worst threat, since the copy that has to survive ransomware must outlast every other copy in the retention set The size of the dataset, because capacity on the backup target is what bounds how many copies can be kept The RPO target, since it fixes the interval between copies and therefore how many exist in any window The detection delay of the slowest-detected threat — usually application corruption, which can sit undetected for weeks
Q2. It is acceptable for the backup process to hold write access to the backup target.

True False
Q3. Match controls to threats.

A cluster has nightly RBD snapshots retained for seven days and no off-cluster copies. The team wants to know what is covered.
Q4. What must a copy be outside of, to survive a compromised application credential?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Set retention from the detection delay of the slowest-detected threat — usually application corruption, not the dramatic ones. Ensure the backup process cannot delete or overwrite what it wrote, or a compromise of that process reaches the backups too.

Cross-course references

Kubernetes: an operator with delete permission on backups extends every blast radius
Linux: retention is bounded by how long a problem can hide, not how bad it is