Lab: LVM snapshot for safe deployment

This lab walks through a snapshot-protected deployment. You will simulate a deployment that modifies data on an LVM volume, then roll back if it fails.

Objective

By the end of this lab, you can:

Take an LVM snapshot before a deployment.
Deploy a change to the volume.
Detect the deployment failure.
Roll back the change via snapshot merge.
Verify the data is intact after the rollback.

Tasks

Task 1: Set up the working volume

Use the loop device and LVM stack from the previous lab, or create a fresh one:

truncate -s 1G /tmp/snap-lab.img
LOOP=$(losetup --find --show /tmp/snap-lab.img)
pvcreate $LOOP
vgcreate vg_snap $LOOP
lvcreate -L 500M -n data vg_snap
mkfs.ext4 /dev/vg_snap/data
mkdir /mnt/snap
mount /dev/vg_snap/data /mnt/snap
echo "v1" > /mnt/snap/version
echo "data file" > /mnt/snap/data.txt
cat /mnt/snap/version /mnt/snap/data.txt

The data on the volume is “v1” and a data file.

Task 2: Take a snapshot before deployment

# snapshot before the change
lvcreate -L 50M -s -n data-pre-deploy /dev/vg_snap/data

Task 3: Simulate a deployment that modifies data

# simulate a successful deployment: v2
umount /mnt/snap
mount /dev/vg_snap/data /mnt/snap
echo "v2" > /mnt/snap/version
echo "new data file" > /mnt/snap/data.txt
cat /mnt/snap/version /mnt/snap/data.txt

The volume now has “v2” and a new data file.

Task 4: Detect the failure and roll back

Imagine the deployment failed. The version should be v2 but the data file is corrupted:

# corrupt the data
echo "CORRUPT" > /mnt/snap/data.txt
cat /mnt/snap/data.txt
# roll back to the snapshot
umount /mnt/snap
lvconvert --merge /dev/vg_snap/data-pre-deploy
mount /dev/vg_snap/data /mnt/snap
echo "after rollback:" $(cat /mnt/snap/version /mnt/snap/data.txt)

The data should be back to “v1” and the original data file.

Task 5: Document

Write a runbook entry that the operations team can use for snapshot-protected deployments on production hosts. The runbook should cover:

When to take a snapshot (always before a deployment, before a migration, before any risky change).
The snapshot size to allocate (5-10% of the LV is usually enough for short-lived deployment snapshots).
How to verify the change before removing the snapshot (smoke tests, integrity checks).
How to merge the snapshot (lvconvert —merge) or remove it (lvremove) on success.

Validation

A snapshot was taken before the deployment.
The deployment modified the data.
The corruption was detected and the snapshot was merged.
The data is back to the pre-deployment state.
A runbook entry was written for the operations team.

Cleanup

umount /mnt/snap
lvremove vg_snap/data
vgremove vg_snap
losetup -d $LOOP
rm -f /tmp/snap-lab.img
rmdir /mnt/snap 2>/dev/null

What you learned

LVM snapshots provide a cheap, instant safety net for any deployment that modifies production data.
The workflow is: take snapshot, make change, verify, either merge (keep change) or remove snapshot (discard change).
For production, a snapshot is the first line of defense; a backup is the second.

Objectives

Prerequisites