Block-level images: use cases and limits

Rolling a filesystem back to a snapshot closed the previous part with a recovery that works by discarding, and every mechanism up to it has worked inside a filesystem, on objects that have names. This part drops below that line. A block-level image copies a device as an ordered run of sectors, with no idea which of them hold a file, which hold a partition table and which hold nothing. That property is why an image does something no file-level backup can, and it is the whole list of reasons it costs what it costs.

What an image reproduces that a file walk never names

A file-level backup enumerates a mounted filesystem and copies what it finds: thorough about everything the filesystem will name, silent about everything else. The apparatus that makes a machine boot is in the second category. The partition table sits at the front of the device, and in the GPT layout a second copy of its header sits at the last addressable block, so the table describes a device of one particular sector count. The gap before the first partition holds a BIOS bootloader’s later stages. Inside each partition, superblocks, bitmaps, journals and inode tables turn sectors into named files, and are not themselves files.

An image reproduces all of it precisely because it is not looking at any of it. It reads sector zero through sector n and writes them out; the bootloader comes along because it occupies those addresses. What lands at the far end is the device, not a reconstruction of it.

The second property is stated less often: the restore does not depend on the backup tool being able to interpret the filesystem. A file-level tool must mount and walk, so it needs a driver, a readable superblock and a traversable filesystem; an imager needs none of those, and can capture a format nobody has documented or a filesystem damaged in exactly the way you want preserved. Those two properties are the whole legitimate case for imaging, and every limit below is one of them seen from the cost side.

The device cannot tell you which blocks matter

If the copier does not parse the filesystem, it cannot know whether a sector belongs to a live file, a deleted one, or a region never written, so the honest default is to copy all of them: an image is sized by the device, not by what is stored on it. The archive-fidelity capture measures that accounting at container scale, on a file with an apparent size of 200 MiB and no blocks allocated.

$ tar -cf naive.tar src

  [source]
  sparse.img apparent    : 200M
  sparse.img allocated   : 0

$ tar -xf naive.tar -C r1
archive size: 201M
[restored from default tar]
  sparse.img apparent    : 200M
  sparse.img allocated   : 200M

Nothing was lost, and a checksum of the contents would pass. What changed is that a region costing nothing now costs 200M, and the artefact between them measured 201M. The same source, through a copier that knows what a hole is:

$ tar --acls --xattrs --xattrs-include='*' --sparse -cf full.tar src

  archive size: 100K
[restored from tar --acls --xattrs --sparse]
  sparse.img apparent    : 200M
  sparse.img allocated   : 0

201M against 100K, decided by whether the tool was told to represent absence rather than transcribe it. Move the exponents and the shape is identical on a device: a 900 GiB volume holding 60 GiB of files costs 900 GiB of stored image and 900 GiB of read and write work, nightly.

Two levers change that, and only together. A copier that detects runs of zeros and seeks over them instead of writing them — dd conv=sparse, as the coreutils manual describes — and free space that genuinely reads as zeros, which is what fstrim arranges by reporting the filesystem’s unused ranges to the device as discarded. Deleted contents are not zeros; they are the old data where it was left.

MNT=/srv/data
SNAP=/dev/rbdrvg/data_snap
IMG=/backup/data-0900.img

fstrim -v "$MNT"
dd if="$SNAP" of="$IMG" bs=4M conv=sparse status=progress
du -h --apparent-size "$IMG"
du -h "$IMG"

The two du readings are the check: apparent size reports the device, allocated size reports what the image cost.

An image of a live device is a smear, not a moment

The second cost is time. Imaging means reading a device end to end — minutes for a small volume, hours for a large one — and on a running system writes land behind and ahead of the read throughout.

The result corresponds to no instant. Sector 12 was read at the start and sector 12 million near the end, so a transaction touching both is represented by half of itself, and a journal read early can refer to blocks rewritten later. Part IV drew the line between crash-consistent and application-consistent state, and this fails the first bar: a crash-consistent image is what a power cut would have left, and a power cut at least happens at one instant.

The fix is the previous part’s mechanism used as a stage. Snapshot first, then image the snapshot, which no longer moves while you read it.

$ md5sum /mnt/snap/orders.csv

origin  orders.csv now:
ORDER-9999,0.00
snapshot orders.csv still:
ORDER-1001,4500.00
ORDER-1002,1250.00
md5 read from snapshot : 9eb4e2ad8e08e1dcaaf87ababab964b0
md5 recorded at 09:00  : 9eb4e2ad8e08e1dcaaf87ababab964b0
MATCH - the snapshot still holds the 09:00 ledger

That yields an image of a real moment, not an application-consistent one. What the snapshot froze is whatever the applications had committed to disk at 09:00, half-written records included. Clearing that bar requires quiescing before the snapshot; the imaging step changes nothing about it either way.

Restoring: sector counts, key material and unfamiliar hardware

An image is written back as an ordered run of sectors, so everything the target does not share with the source becomes a problem at that moment.

Size is a hard floor. The image describes a device of a particular sector count: the GPT secondary header belongs at the last addressable block, partition entries name absolute ranges, and each filesystem records its own size in its superblock. A target at least as large works; a smaller one cannot receive the tail of the image, however little of it holds data. A larger one takes the image and then needs the leftovers dealt with, because the secondary header no longer sits where the primary says. Check first:

IMG=/backup/data-0900.img
TARGET=/dev/sdb

if [ "$(blockdev --getsize64 "$TARGET")" -lt "$(stat -c '%s' "$IMG")" ]; then
  printf 'refusing: %s is smaller than %s\n' "$TARGET" "$IMG" >&2
  exit 1
fi

Ciphertext restores as ciphertext. An image of an encrypted device is a faithful copy of encrypted bytes and nothing more; restored without the passphrase or key file, it unlocks for nobody. The key material therefore has to survive the event the image protects against and be recoverable from somewhere that is not the host being imaged, which is Part IX’s subject. The LUKS header with its key slots lives at the front of the device, so an image starting after it carries no way to open anything, and cryptsetup luksHeaderBackup exists because that header is small and irreplaceable.

Different hardware turns identity into a fault. An initramfs assembled with only the storage drivers the old controller needed will not find the new one, and the boot stops there. Predictable interface names derive from firmware and bus topology, so the restored network configuration names an interface that does not exist here; filesystem and volume UUIDs are copied verbatim, so two devices can claim one identity while the source is still online.

Where an image is the right answer, and where it is not

Reach for an image when you need whole-system reproduction — the state that makes a machine boot and be itself, which a file-level backup never enumerates — or when the filesystem is something your tools cannot read, because copying without interpreting is the property nothing else has.

Use file-level and application-aware backups for everything else, which is most things. They cost what the data costs rather than what the device costs, they restore one file without staging a volume, and an application-aware backup captures a state the application will accept rather than merely survive.

The usual production answer is both: image the system disk when the machine is built and after significant change, and back the data up at file or application level on its own schedule.

Production discipline

1. Size an image from the device, not from the data on it. A copier with no handling for unwritten regions produced 201M from a source whose 200M region showed allocated : 0, and restored it with allocated : 200M.
2. Give the copier something it is allowed to skip. Trim or zero-fill the free space first, then use a sparse-aware copy such as dd conv=sparse, and record both du readings so the day the allocated size reaches the apparent size is a day someone notices it.
3. Image a frozen device, never a live one. Snapshot first: in the LVM capture the snapshot still returned 9eb4e2ad8e08e1dcaaf87ababab964b0 after the origin was truncated. If the image must be application-consistent, quiesce before the snapshot.
4. Never leave key material inside the thing it protects. An image of an encrypted device is ciphertext, so escrow the passphrase and a luksHeaderBackup in a different failure domain, and confirm the imaged range includes the header.
5. Prove the image boots on the hardware you would restore it to. A verified copy of the bytes says nothing about drivers, interface names or duplicate UUIDs. Restore to a spare device or a VM and check that the service answers.

Cross-course references

• Linux for Production Sysadmins — Part XIII (Disks and Block Devices) establishes the sector-addressed model this lesson copies wholesale, and it is why an imaging tool cannot report which blocks are in use: that interface carries no allocation state, so the 201M-against-100K difference is decided by what the copier infers. Part IX (Boot Process) covers the bootloader and initramfs that decide whether a restored image boots or merely exists.
• Proxmox VE for Production Operators — Part IX (Virtual Machines) is where a block image stops being an exceptional artefact and becomes the normal unit of storage, because a VM disk is one. Every limit measured here reappears redistributed: identical virtual controllers largely remove the driver problem, while sparse sizing and guest quiescing become settings you choose.
• Ceph & Distributed Storage for Production Sysadmins — Part XXXV (RBD Architecture) describes a block image whose objects exist only where something was written, so the sparse-awareness question measured here is answered by the storage layer rather than the copying tool; Part CVI (RBD Backup) covers exporting such an image, where this lesson’s crash-consistency reasoning applies to the snapshot it is taken from.

Quiz