Skip to main content
RunBook Academy

← All checklists in Ansible

As neededDeployment

Checklist: Rolling deployment

19 items ·15 critical ·3 warn ·1 info

Run this before a change is rolled across a tier, after the canary has been validated. It is the last review of the mechanics: batch size, thresholds, drain, health gating and what happens if you stop halfway.

Most items are static reads of the playbook. The three that connect to hosts - the delegated-target ping, the host count and the task listing - read only.

Three behaviours verified by execution

The claims in this checklist about serial, max_fail_percentage and run_once were established by running them on ansible-core 2.21.3, not read from documentation, because all three surprise people:

A play with serial: 2 across four hosts, whose first batch contains a failing host and which declares no threshold, continued - it dropped the failed host and changed the remaining three in the following batches. Adding max_fail_percentage: 0 stopped it after the first batch.

With four hosts and one failure, max_fail_percentage: 25 did not stop the play. The comparison is strictly greater-than, so 25% failed does not exceed a threshold of 25.

A run_once task under serial: 2 across four hosts executed twice - once per batch, not once per play.

What a failure means

Every critical item here describes a way the roll can keep going when it should have stopped, or stop in a state nobody planned for. A rolling deployment is a controlled failure mechanism: the control is the batch size, the threshold and the gate between batches, and a defect in any of the three turns the roll back into a simultaneous change with extra steps.

Access this needs

The playbook scans need a checkout. The delegated-target ping and the host count connect to hosts and read. Nothing here changes anything - this is a review, and the roll itself is the next action, not part of this list.

Where the evidence goes

Record the batch size, the threshold and the reasoning behind both in the change record. They are the two numbers a post-incident review will ask about, and “it was 25%” is only useful next to “because we can lose a quarter of the tier and still serve peak”.

Sign-off

  • Roll owner: _______________ Date: ___________
  • Capacity confirmed by: ____ Date: ___________

Critical15 items

  3. python3 - <<'PY'
import yaml
for p in yaml.safe_load(open('playbooks/deploy.yml')) or []:
    if isinstance(p, dict) and 'hosts' in p:
        print(f"play {p.get('name')!r}: hosts={p.get('hosts')} "
              f"serial={p.get('serial', 'NONE')} "
              f"max_fail_percentage={p.get('max_fail_percentage', 'NONE')} "
              f"any_errors_fatal={p.get('any_errors_fatal', False)}")
PY
  4. python3 - <<'PY'
import yaml
for p in yaml.safe_load(open('playbooks/deploy.yml')) or []:
    if not isinstance(p, dict) or 'hosts' not in p: continue
    if 'max_fail_percentage' not in p and not p.get('any_errors_fatal'):
        print(f"FINDING: play {p.get('name')!r} will not stop on its own")
PY
  5. grep -rn 'max_fail_percentage' playbooks/
  6. grep -rn -B2 -A6 'delegate_to' playbooks/
  7. grep -rn 'wait_for' playbooks/ roles/
  9. ansible-playbook -i inventories/production playbooks/deploy.yml --list-tasks | tail -15
  10. grep -rn 'flush_handlers' playbooks/ roles/
  11. grep -rn 'run_once' playbooks/ roles/
  12. ansible -i inventories/production web --list-hosts | tail -n +2 | wc -l

Warning3 items

  1. ansible-config dump | grep '^DEFAULT_FORKS'; grep -rn 'serial:' playbooks/
  2. ansible -i inventories/production loadbalancers -m ansible.builtin.ping --one-line | grep -v 'SUCCESS'
  3. ansible-playbook -i inventories/production playbooks/deploy.yml --list-tasks | grep -c '^ '

Info1 item

  1. grep -rn -A1 'serial:' playbooks/