Crash consistency versus application consistency

Advanced⏱ ~18 minrbdfsfreezevirsh

What you'll learn

Distinguish crash from application consistency
Identify what each workload needs
Quiesce an application before a snapshot
Verify a snapshot is restorable

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

A snapshot that restores into a database that will not start is not a backup, and it looks identical to one that works until it is tried.

The two levels

Crash consistent: the snapshot is what the disk would look like if
power had been cut at that instant.

Application consistent: the application had flushed its state and knew
the snapshot was being taken.

Property	Crash consistent	Application consistent
Filesystem journal replay needed	yes	no
In-flight writes present	partially	no
Application recovery on start	yes	no
Guaranteed restorable	for journalling filesystems, usually	yes
Cost to take	none	quiescing the application

# Substitute your own values before running:
POOL=rbd-vms
IMAGE=vm-disk-01

rbd snap create "$POOL/$IMAGE@crash-consistent"

An RBD snapshot on its own is crash consistent. RBD does not know what
the guest is doing.

What each workload needs

Workload	Needs
A stateless web server’s root disk	crash consistent is fine
A journalling filesystem holding files	crash consistent, journal replays
PostgreSQL, MySQL with proper durability	crash consistent usually works; recovery runs on start
A database with `fsync` disabled for speed	application consistent required
A multi-disk database with data and log separated	application consistent required — the disks must agree
An application with in-memory state	application consistent, or accept the loss

The multi-disk case is the one that fails silently: two images
snapshotted a second apart produce a database whose log and data
disagree.

# a consistency group across images, so they snapshot together
# Substitute your own pool before running:
POOL=rbd-vms

rbd group create "$POOL/dbgroup"
rbd group image add "$POOL/dbgroup" "$POOL/data01"
rbd group image add "$POOL/dbgroup" "$POOL/log01"
rbd group snap create "$POOL/dbgroup@consistent"

Quiescing before a snapshot

# filesystem level, from inside the guest
# Substitute your own values before running:
POOL=rbd-vms
IMAGE=vm-disk-01

fsfreeze -f /var/lib/postgresql
rbd snap create "$POOL/$IMAGE@app-consistent"
fsfreeze -u /var/lib/postgresql

The freeze window must be short. A frozen filesystem blocks writes, and
the snapshot itself is instant — so the window is the round trip, not
the snapshot duration.

# libvirt coordinates the guest agent to freeze filesystems
DOMAIN=vm-db-01
POOL=rbd-vms
IMAGE=vm-disk-01
virsh domfsfreeze ${DOMAIN}
rbd snap create ${POOL}/${IMAGE}@app-consistent
virsh domfsthaw ${DOMAIN}

# database level, where the database supports it
# Substitute your own pool before running:
POOL=rbd-vms

psql -c "SELECT pg_backup_start('snap');"
rbd snap create "$POOL/pgdata@app-consistent"
psql -c "SELECT pg_backup_stop();"

Verifying restorability

The only verification that means anything is restoring and starting the
application.

# Substitute your own values before running:
POOL=rbd-vms
IMAGE=vm-disk-01

rbd clone "$POOL/$IMAGE@app-consistent" "$POOL/restore-test"
rbd map "$POOL/restore-test"
mount "/dev/rbd/$POOL/restore-test" /mnt/restore-test
# start the application against it and confirm it comes up

# the same pool declared in the clone step above
POOL=rbd-vms

umount /mnt/restore-test
rbd unmap "/dev/rbd/$POOL/restore-test"
rbd rm "$POOL/restore-test"

Quiz

Knowledge check · 4 questions

Q1. Why is a crash-consistent snapshot usually sufficient for a database?

Databases do not require a consistent image at all, because the recovery path rebuilds every table from the write-ahead log on each start regardless of what the snapshot caught RBD snapshots are application-aware: librbd signals the guest to flush and quiesce before the snapshot point is taken A database honouring fsync writes its log durably before acknowledging, so a crash-consistent snapshot is exactly a power loss and its recovery path handles it The filesystem journal replays on mount and restores any transaction the database had not yet committed into its own tables
Q2. Snapshotting two RBD images a second apart is adequate for a database with separate data and log volumes.

True False
Q3. Design snapshots for a database workload.

A PostgreSQL instance runs in a VM with its data directory and WAL on separate RBD images. Nightly snapshots are taken of each image in sequence.
Q4. How long is the freeze window when quiescing a filesystem for a snapshot?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Use rbd group snap create for any workload spanning multiple images — sequential snapshots of a data and log pair produce a state the database may refuse to start from. Verify by cloning a snapshot and actually starting the application against it; nothing else verifies anything.

Cross-course references

Kubernetes: VolumeSnapshot groups exist for multi-volume application consistency
Linux: crash consistency is sufficient exactly where the application assumed power loss