Skip to main content
RunBook Academy

← All checklists in Ansible

Monthlypatching

Checklist: Fleet patching with Ansible

21 items ·16 critical ·4 warn ·1 info

Run this every patch cycle. The pre-run items go before the window opens; the post-verify items go immediately after each batch, not at the end of the month.

The commands are written for Debian and Ubuntu targets. On EL hosts substitute dnf check-update, dnf repolist, dnf versionlock list, needs-restarting and rpm -qa for their apt equivalents - the checks are the same, the tools differ.

What a failure means

Patching has a failure mode the other checklists do not: it can report complete success while changing nothing. A held package, a running service that was never restarted, a kernel installed but not booted - all three produce a green recap and a host that is still vulnerable.

That is why so much of this list is post-verification. The critical items before the run are about not breaking the fleet; the critical items after it are about the patch having actually happened.

Access this needs

Most commands here go through the command module and read: apt-get -s simulates without changing anything, apt-cache, apt-mark showhold, needrestart -b and dkms status report state. They need become on most hosts. The fact modules - setup, package_facts, service_facts, stat, find - read only.

The command module will run whatever you hand it. Every invocation in this list is read-only as written; check any variation you make before running it against a fleet.

Where the evidence goes

The before-and-after package_facts output is the patch record. Keep both, per host, with the run timestamp. It is what answers the scanner when the scanner disagrees, and it is the only artefact that distinguishes “we patched it” from “we ran the patch play against it”.

Sign-off

  • Patch owner: ______________ Window: __________
  • Post-verify by: ___________ Date: ___________

Critical16 items

  2. ansible -i inventories/production web -m ansible.builtin.command -a 'apt-get -s dist-upgrade' --one-line
  3. ansible -i inventories/production web -m ansible.builtin.command -a 'apt-cache policy' --one-line
  4. ansible -i inventories/production web -m ansible.builtin.find \
  -a 'paths=/etc/apt/keyrings,/etc/apt/trusted.gpg.d patterns=*.gpg,*.asc' --one-line
ansible -i inventories/production web -m ansible.builtin.command --one-line \
  -a 'grep -rn trusted=yes /etc/apt/sources.list /etc/apt/sources.list.d/' | grep -v 'rc=1'
  5. ansible -i inventories/production web -m ansible.builtin.command -a 'apt-mark showhold' --one-line
  7. ansible -i inventories/production web -m ansible.builtin.setup -a 'filter=ansible_mounts' --one-line
  8. ansible -i inventories/production web -m ansible.builtin.find -a 'paths=/var/backups patterns=*.tar.gz age=-1d' --one-line | grep -v '"matched": [1-9]'
  9. ansible -i inventories/production web01.example.com -m ansible.builtin.command -a 'apt-cache madison openssl' --one-line
  10. ansible -i inventories/production web -m ansible.builtin.stat -a 'path=/var/run/reboot-required' --one-line
  11. ansible -i inventories/production web -m ansible.builtin.command -a 'needrestart -b' --one-line
  12. ansible -i inventories/production web -m ansible.builtin.command -a 'dkms status' --one-line
  13. ansible -i inventories/production etcd --list-hosts; grep -rn 'serial:' playbooks/patch.yml
  14. python3 - <<'PY'
import yaml
for p in yaml.safe_load(open('playbooks/patch.yml')) or []:
    if not isinstance(p, dict) or 'hosts' not in p: continue
    if not p.get('serial'):
        print(f"FINDING: play {p.get('name')!r} patches without batching")
    if 'max_fail_percentage' not in p and not p.get('any_errors_fatal'):
        print(f"FINDING: play {p.get('name')!r} patches without a failure threshold")
PY
  15. ansible -i inventories/production web -m ansible.builtin.package_facts --one-line
  16. ansible -i inventories/production web -m ansible.builtin.setup -a 'filter=ansible_uptime_seconds,ansible_kernel' --one-line

Warning4 items

  1. ansible -i inventories/production web -m ansible.builtin.service_facts --one-line | grep -o 'unattended-upgrades[^,]*'
  2. ansible -i inventories/production web -m ansible.builtin.package_facts --one-line

Info1 item