This checklist applies to every production host. The five items marked critical are non-negotiable; the remaining items document the operational maturity of the boot recovery process.
Procedure
For each item:
- Read the requirement.
- Confirm the host satisfies it.
- For automated checks, run the listed command and confirm the expected output.
- Document exceptions and remediation plans.
The critical items are pre-requisites for any boot recovery. A host that lacks OOB access cannot be recovered remotely.
Emergency-shell access is on that list because OOB access
alone is not enough. systemd hands the emergency and rescue
consoles to sulogin, which demands the root password —
and Ubuntu Server, Debian cloud images and most golden
images ship with the root account locked, so sulogin
refuses and loops. You reach the console, and the console
will not let you in. Either set a root password or install a
drop-in carrying Environment=SYSTEMD_SULOGIN_FORCE=1 on
both units, and test it, because init=/bin/bash on the
GRUB line is a workaround you do not want to be learning at
03:00.
Persistent journal storage is on that list for the same
reason. Storage=auto, the default on most images, uses
/var/log/journal/ only if the directory already exists — it
never creates it. Where it does not exist, journald runs in
tmpfs under /run/log/journal/ and every line is destroyed at
reboot, so after a boot failure journalctl -b -1 returns
Data from the specified boot is not available. That is a
build-time decision: it cannot be applied retrospectively once
the host has rebooted, which is why it belongs here and not in
the incident.
Approval
A host’s boot-recovery readiness is reviewed quarterly. The checklist is signed off by the operations team. Items marked critical must all be satisfied for the host to be considered production-ready.