Tiering copies for two different recovery times

Intermediate⏱ ~18 mincephrbdrclone

What you'll learn

Separate the frequent restore from the rare one
Compute the offsite tier restore time from link bandwidth
Account for retrieval latency and egress cost
Commit two distinct RTOs rather than one

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

The restore you will perform this quarter is one image or one bucket prefix. The restore you are designing for is the whole estate. Sizing one architecture for both makes the common case slow and the rare case unaffordable.

Two tiers, two jobs

Tier	Job	Restores served	Target RTO
local, outside the cluster’s control plane	the frequent restore	single image, file, bucket, prefix	minutes to an hour
offsite, independently operated	site or cluster loss	everything	days, stated honestly

ceph df
rbd -p rbd du | tail -5

The offsite RTO is arithmetic

python3 - <<'EOF'
size_tib   = 40
link_gbps  = 10
util       = 0.6          # what you actually get, not the line rate
secs = size_tib * 1024**4 * 8 / (link_gbps * 1e9 * util)
print("%.1f hours" % (secs / 3600))
EOF

Data	1 Gb/s at 60%	10 Gb/s at 60%	25 Gb/s at 60%
4 TiB	16 h	1.6 h	0.7 h
40 TiB	6.8 days	16 h	6.5 h
400 TiB	68 days	6.8 days	2.7 days

No backup tool changes these numbers. If the offsite tier is the only
tier, the row your estate lands on is your RTO, and it is measured in
days.

What retrieval class adds

Class	First byte	Restore of 40 TiB
local disk or object target	seconds	bandwidth-bound
standard cloud object storage	seconds	bandwidth plus egress cost
archive or cold class	hours	retrieval latency then bandwidth

rclone lsjson remote:acme-backup/vm-104 --stat
rclone copy remote:acme-backup/vm-104.img /restore/ --transfers 16 --progress

Egress is a real line item: 40 TiB retrieved at typical per-GiB egress pricing costs thousands, once, at the worst possible moment. Budget it or the decision to restore becomes a procurement conversation.

The local tier only counts if it is independent

# a local target that is not this cluster, this rack, or this credential domain
rbd export rbd/vm-104 - | ssh backup-01 'cat > /srv/backup/vm-104.img'
rclone copy /srv/backup/ offsite:acme-backup/ --transfers 8

Quiz

Knowledge check · 4 questions

Q1. What sets the recovery time of an offsite backup tier?

The backup software's compression ratio, since a well-compressed archive crosses the same link in a small fraction of the raw transfer time it would need The link bandwidth and the data volume — 40 TiB over a 10 Gb/s link at realistic utilisation is roughly sixteen hours before anything starts The number of parallel transfer streams, because one stream cannot saturate a long fat link while eight or more will fill it completely The object count in the backup, since per-object request overhead dominates the transfer and does not shrink as the objects get larger
Q2. A single well-designed backup tier can serve both the monthly single-image restore and the once-a-decade cluster loss.

True False
Q3. Design backup tiers against a stated RTO.

An estate holds 60 TiB of RBD data. The business states a four-hour RTO. The proposed design is nightly copies to cloud object storage in an archive class over a 1 Gb/s uplink.
Q4. When does a local backup target fail to count as a second copy?

Passing score: 75%. Answers are checked in this browser.

Production discipline

State two recovery times, one per scenario, and publish the offsite figure as the arithmetic gives it — hiding a six-day transfer behind a four-hour commitment fails at the worst moment. Verify the local tier sits outside the cluster’s failure and credential domain, or it is not a tier.

Cross-course references

Kubernetes: cluster-wide restore and single-namespace restore are different products of the same tool
Linux: bandwidth-bound recovery times are arithmetic, not an implementation detail