Lab: LVM create, extend, and snapshot

This lab walks through the full LVM lifecycle: create, use, extend, snapshot, rollback. By the end you will have created and operated a complete LVM stack.

Objective

By the end of this lab, you can:

Create a PV/VG/LV from a loop device.
Create and mount a filesystem on the LV.
Extend the LV and grow the filesystem online.
Take a snapshot, make a destructive change, and roll back via merge — then prove the rollback with checksums.
Exhaust an undersized snapshot and recognise an invalidated one on sight.
Shrink an LV safely, and recognise the wrong order on sight.

Blast radius

Tasks

Task 1: Create the stack

Create a 1 GB file, turn it into a loop device, and build the LVM stack on it:

truncate -s 1G /tmp/lvm-lab.img
LOOP=$(losetup --find --show /tmp/lvm-lab.img)
echo "Loop device: $LOOP"
pvcreate $LOOP
vgcreate vg_lab $LOOP
lvcreate -L 200M -n data vg_lab
mkfs.ext4 /dev/vg_lab/data
mkdir /mnt/lab
mount /dev/vg_lab/data /mnt/lab
df -h /mnt/lab

Verify the stack with pvs, vgs, lvs.

Now put real, identifiable data on the volume. Task 3 rolls back to this state, and a rollback you cannot measure is not a rollback you can trust:

for i in $(seq 1 5); do echo "original content $i" > /mnt/lab/file$i.txt; done
sha256sum /mnt/lab/*.txt | tee /tmp/pre-snapshot.sha256
ls -l /mnt/lab
sync

The checksum file lives on /tmp, outside the volume, so the destructive step in Task 3 cannot take the evidence with it.

Task 2: Extend the LV online

# add a second loop device and extend the VG
truncate -s 1G /tmp/lvm-lab-2.img
LOOP2=$(losetup --find --show /tmp/lvm-lab-2.img)
vgextend vg_lab $LOOP2
# extend the LV by 100M
lvextend -L +100M /dev/vg_lab/data
# grow the filesystem
resize2fs /dev/vg_lab/data
df -h /mnt/lab

The LV should be 300 MB and the filesystem should match.

Task 3: Take a snapshot, modify, roll back

# snapshot before the change
lvcreate -L 50M -s -n data-pre-mod /dev/vg_lab/data
lvs -o lv_name,origin,lv_size,data_percent,lv_attr vg_lab

# make a destructive change - note the trailing /*.txt
rm -rf /mnt/lab/*.txt
ls -l /mnt/lab                       # expect: no .txt files remain
sha256sum -c /tmp/pre-snapshot.sha256 || echo "data is gone, as intended"

# roll back
umount /mnt/lab
lvconvert --merge /dev/vg_lab/data-pre-mod
mount /dev/vg_lab/data /mnt/lab

# the only result that counts
sha256sum -c /tmp/pre-snapshot.sha256

The final sha256sum -c must print OK for all five files. Anything else — a missing file, a FAILED line, a non-zero exit — means the rollback did not restore the volume, and the task is not complete. “The directory looks right” is not a verification; a matching checksum is.

Destroy the contents, never the mount point

The destructive step is rm -rf /mnt/lab/*.txt, not rm -rf /mnt/lab.

Aimed at the mount point itself, rm -rf deletes everything inside the mounted filesystem and then fails on the mount point directory with EBUSY, because it is an active mount. You get an error and an ambiguous result: some data destroyed, the command reporting failure, no clean before-and-after to reason about.

The habit is worse than the error. Carry rm -rf <mountpoint> into production and run it on a host where that volume happens to be unmounted — after a failed boot, or before the mount unit has started — and you delete the directory and whatever the parent filesystem is holding underneath it. The mount was the only thing protecting you. Target the contents, and prefer naming a pattern (*.txt) over a bare *.

Task 3b: Fill a snapshot until it dies

A snapshot is not free storage. It holds copies of the origin’s original chunks, made on demand as the origin is written. Run out of snapshot space and the snapshot does not shrink, throttle or warn — it is dropped, permanently, and everything that depended on it is gone.

# a deliberately undersized snapshot
lvcreate -L 20M -s -n data-toosmall /dev/vg_lab/data

# write far more to the origin than the snapshot can hold
dd if=/dev/urandom of=/mnt/lab/fill bs=1M count=60 oflag=direct
sync
lvs -o lv_name,origin,lv_size,data_percent,lv_attr vg_lab

Read the Attr column for data-toosmall. It now begins with I for Invalid, and Data% is blank rather than a percentage. The snapshot is dead. Prove that it is not a rollback path any more:

lvconvert --merge /dev/vg_lab/data-toosmall   # expect: failure
lvremove -f vg_lab/data-toosmall
rm -f /mnt/lab/fill
sync

lvconvert refuses to merge an invalid snapshot. There is no recovery and no partial rollback — the copy-on-write store is incomplete, so LVM cannot reconstruct the origin’s old state at all.

This is the failure mode that turns a routine change window into an incident: the snapshot is taken as the rollback plan, the change writes more than the snapshot was sized for, the snapshot silently invalidates, and the rollback is discovered to be impossible only when it is needed. Size a snapshot for the volume of writes you expect during the change window, not for the size of the data you care about, and monitor Data% while the change runs.

Task 4: Shrink the LV - filesystem first, LV second

Extending runs LV first, then filesystem. Shrinking runs the other way round. Get the order wrong and the filesystem is larger than the device holding it, which is unrecoverable.

# 1. Gate: what filesystem is this? ext4 shrinks offline; XFS never shrinks.
lsblk -f /dev/vg_lab/data
df -h /mnt/lab

# 2. Gate: how small can it go? -P only reports, it changes nothing.
umount /mnt/lab
resize2fs -P /dev/vg_lab/data

# 3. Mandatory check. resize2fs refuses to shrink an unchecked filesystem.
e2fsck -f /dev/vg_lab/data

# 4. FILESYSTEM first, to below the size the LV will become.
resize2fs /dev/vg_lab/data 120M

# 5. Dry-run the LV change, then do it. 150M is LARGER than the 120M filesystem.
lvreduce -t -L 150M /dev/vg_lab/data
lvreduce -L 150M /dev/vg_lab/data       # answer y at the prompt

# 6. Reclaim the safety margin, remount, verify.
resize2fs /dev/vg_lab/data
mount /dev/vg_lab/data /mnt/lab
lvs vg_lab
df -h /mnt/lab

The LV should be 150 MB and the filesystem should now match it. The 30 MB gap in step 5 is deliberate: LVM allocates in whole physical extents and resize2fs works in whole filesystem blocks, so an apparently equal size can round the LV below the filesystem.

Task 5: Observe the wrong order failing

This step deliberately corrupts a throwaway LV so you can see what the wrong order actually produces. It runs on vg_lab only, on a separate LV that exists purely to be destroyed.

lvcreate -L 200M -n wrongway vg_lab
mkfs.ext4 /dev/vg_lab/wrongway

# The wrong order: shrink the LV while the filesystem still
# believes it owns 200M. -f skips the confirmation prompt,
# which is the exact safety net you must never skip in production.
lvreduce -f -L 60M /dev/vg_lab/wrongway

# The damage is already done. e2fsck reports it:
e2fsck -f /dev/vg_lab/wrongway

e2fsck reports that the filesystem size according to the superblock is larger than the physical size of the device, and warns that the superblock or the partition table is likely corrupt. There is no repair for this. In production the fix is a restore from backup.

Note what did not happen: lvreduce did not inspect the filesystem and did not fail. It printed a warning, took the y, and reported success. Without -r, LVM has no idea a filesystem is there.

lvremove -f vg_lab/wrongway

Task 6: Clean up

umount /mnt/lab
lvremove -f vg_lab/wrongway 2>/dev/null || true
lvremove -f vg_lab/data-toosmall 2>/dev/null || true
lvremove vg_lab/data
vgremove vg_lab
pvremove $LOOP $LOOP2
losetup -d $LOOP $LOOP2
rm -f /tmp/lvm-lab.img /tmp/lvm-lab-2.img /tmp/pre-snapshot.sha256
rmdir /mnt/lab

Validation

A complete LVM stack was created and torn down.
The LV was extended online with the filesystem.
Five files were written and checksummed to /tmp/pre-snapshot.sha256 before the snapshot was taken.
A snapshot was taken, the files were deleted, and after lvconvert --merge the command sha256sum -c /tmp/pre-snapshot.sha256 printed OK for all five files and exited zero. This is the pass condition for the rollback — not the appearance of the directory listing.
An undersized snapshot was driven to exhaustion, its Attr column was observed starting with I (Invalid), and lvconvert --merge against it was observed failing.
The LV was shrunk with the filesystem resized first, and lvs and df -h agree on the final size.
The wrong order produced an e2fsck size mismatch on the throwaway LV.
The cleanup removed all LVM artifacts.

Cleanup

The Task 6 cleanup restores the host to its starting state: both LVs removed, the VG removed, LVM metadata wiped from the loop devices with pvremove, the loop devices detached, the backing files deleted, and the /mnt/lab mountpoint removed.

Verify with lsblk, losetup -a, and vgs. losetup -a should not list /tmp/lvm-lab*.img, and vgs should not list vg_lab. If a losetup -d fails with “device is busy”, the LV or VG above it was not removed; work back up the list.

What you learned

The PV-VG-LV stack is the foundation of flexible storage on Linux.
Online extend works for both LV and filesystem.
Shrink is offline for ext4, impossible for XFS, and runs in the opposite order to extend: filesystem first, LV second, LV always the larger of the two.
lvreduce does not check the filesystem unless you pass -r. The confirmation prompt is the only safety net, so -f belongs in throwaway labs and nowhere else.
LVM snapshots are the cheap, instant safety net for local changes — but only while they have space. An exhausted snapshot is marked Invalid and cannot be merged, so a rollback plan built on an undersized snapshot fails at the moment it is needed. Size for the writes during the change window, and watch Data%.
A rollback is verified by a checksum, not by a directory listing.
Destructive commands target contents (/mnt/lab/*.txt), never the mount point itself. rm -rf on a mount point fails with EBUSY after emptying the filesystem, and on an unmounted path it deletes the directory from the parent filesystem instead.
Cleanup is just as important as creation; LVM does not release disk space until the LVs and VGs are removed.

Objectives

Prerequisites