Ansible playbook shape — pre-flight, change, post-flight, and the anti-lockout gates

A firewall playbook is not a one-shot script. It is a four-phase workflow: pre-flight checks that confirm the firewall is reachable and the change is safe to apply; the change tasks themselves; post-flight verification that the firewall’s desired state matches the playbook; and a final guarantee that nothing has been left in a partial state. Each phase catches a different class of failure, and skipping any one of them gives the operator a false sense of safety.

This lesson covers the four-phase shape, handlers for controlling service reloads, role-and-task-file structure for reuse, and a worked playbook that adds a rule to a single firewall and then a serial fleet.

The four phases

A production firewall playbook has four phases, in order:

1. Reach. Confirm the firewall is reachable over the API and authentication works. An unreachable firewall or a wrong API key fails the play before any change is attempted.
2. Pre-flight. Confirm the change is safe to apply. Check the existing rules for an anti-lockout rule that would be displaced; check the rule order to confirm the new rule will land in the expected position; check for conflicting rules. A pre-flight failure aborts before any change.
3. Change. Apply the desired state via the Ansible modules. Each task is idempotent (search-then-add-or-set internally); the play reports changed for tasks that actually modified something.
4. Verify. Read the state back and confirm the desired state is present. A verification failure signals a partial change; the rollback path is to revert to the configuration revision recorded before the change began.

The phases map to plays in the playbook:

- name: Reach — API connectivity
  hosts: firewalls
  connection: local
  gather_facts: false
  module_defaults:
    group/ansibleguy.opnsense.all:
      firewall: "{{ ansible_host }}"
      api_key: "{{ opnsense_api_key }}"
      api_secret: "{{ opnsense_api_secret }}"
      ssl_verify: true
  tasks:
    - name: Confirm the API credentials authenticate
      ansible.builtin.uri:
        url: "https://{{ ansible_host }}/api/core/firmware/status"
        method: GET
        url_username: "{{ opnsense_api_key }}"
        url_password: "{{ opnsense_api_secret }}"
        force_basic_auth: true
        status_code: 200

- name: Pre-flight — read the current rules
  hosts: firewalls
  connection: local
  gather_facts: false
  tasks:
    - name: List the automation rules
      ansibleguy.opnsense.list:
        target: rule
      register: current_rules

    - name: Fail if the change ticket already has rules on this firewall
      ansible.builtin.assert:
        that:
          - current_rules.data | selectattr('description', 'search', 'CHG-2026-1314') | list | length == 0
        fail_msg: "CHG-2026-1314 rules already exist here; re-running would need a review first"

- name: Change — apply the desired state
  hosts: firewalls
  connection: local
  gather_facts: false
  tasks:
    - name: Ensure outbound allow for CI runners
      ansibleguy.opnsense.rule:
        description: "CHG-2026-1314 outbound"
        match_fields: ['description']
        action: pass
        interface: ['lan']
        direction: in
        ip_protocol: inet
        protocol: any
        source_net: "{{ ci_runners_alias }}"
        destination_net: any
        log: true
        reload: false
      notify: Reload the filter

- name: Verify — confirm the rule is present
  hosts: firewalls
  connection: local
  gather_facts: false
  tasks:
    - name: Re-read the automation rules
      ansibleguy.opnsense.list:
        target: rule
      register: applied_rules

    - name: Assert exactly one rule carries the ticket
      ansible.builtin.assert:
        that:
          - applied_rules.data | selectattr('description', 'search', 'CHG-2026-1314') | list | length == 1

  handlers:
    - name: Reload the filter
      ansibleguy.opnsense.reload:
        target: rule

Three details in that skeleton do real work.

connection: local is mandatory. The modules run on the controller and reach the firewall over its API; without it Ansible tries to SSH into the firewall and execute Python there, which is not how these modules work.

module_defaults under group/ansibleguy.opnsense.all sets the connection parameters once for every module in the collection, rather than repeating firewall, api_key and api_secret on every task. The credentials come from vault-encrypted variables — never from the playbook.

reload: false on the change task, paired with a handler that reloads once, is the batching decision. Left at its default the module applies after every write, so a play with eight rule tasks reloads the ruleset eight times.

Each play targets the same group with a different intent. The reach play can also be run alone (ansible-playbook reach.yml) as a sanity check before any change is queued.

$ ansible-playbook -i inventories/prod.yml playbooks/firewall-rules.yml --limit fw-dc1-edge-01 --check --diff --ask-vault-pass

PLAY [Reach — API connectivity] ***********************************************
TASK [Confirm the API credentials authenticate] *******************************
ok: [fw-dc1-edge-01]

PLAY [Pre-flight — read the current rules] ************************************
TASK [List the automation rules] **********************************************
ok: [fw-dc1-edge-01]
TASK [Fail if the change ticket already has rules on this firewall] ***********
ok: [fw-dc1-edge-01]

PLAY [Change — apply the desired state] ***************************************
TASK [Ensure outbound allow for CI runners] ***********************************
--- before
+++ after
@@ -1,1 +1,8 @@
-{}
+{
+    "action": "pass",
+    "description": "CHG-2026-1314 outbound",
+    "destination_net": "any",
+    "interface": ["lan"],
+    "source_net": "ci_runners"
+}
changed: [fw-dc1-edge-01]

PLAY [Verify — confirm the rule is present] ***********************************
TASK [Re-read the automation rules] *******************************************
ok: [fw-dc1-edge-01]
TASK [Assert exactly one rule carries the ticket] *****************************
failed: [fw-dc1-edge-01]

PLAY RECAP ********************************************************************
fw-dc1-edge-01    : ok=4  changed=1  unreachable=0  failed=1

Handlers and flush_handlers

A handler is a task that runs only when notified by a change. In a firewall playbook the handler is the reload, and the reason to use one is that every write module in the collection applies by default — reload: true — so a play with eight rule tasks rebuilds and reloads the ruleset eight times unless you say otherwise.

tasks:
  - name: Ensure outbound allow for CI runners
    ansibleguy.opnsense.rule:
      description: "CHG-2026-1314 outbound"
      match_fields: ['description']
      source_net: "{{ ci_runners_alias }}"
      reload: false
    notify: Reload the filter

handlers:
  - name: Reload the filter
    ansibleguy.opnsense.reload:
      target: rule

Handlers run at the end of the play, which is usually what you want: all the writes happen, then one reload. ansible.builtin.meta: flush_handlers forces pending handlers to run at a chosen point instead — for example after the rule tasks but before a verification play that needs the rules to be live.

The rollback shape is block/rescue, and the important part is what the rescue does. Aliases and rules reload separately, so a change-set that touches both notifies both handlers:

tasks:
  - block:
      - name: Ensure the CI runners alias
        ansibleguy.opnsense.alias:
          name: ci_runners
          type: host
          content: "{{ ci_runner_addresses }}"
          reload: false
        notify: Reload the aliases

      - name: Ensure outbound allow for CI runners
        ansibleguy.opnsense.rule:
          description: "CHG-2026-1314 outbound"
          match_fields: ['description']
          source_net: ci_runners
          reload: false
        notify: Reload the filter

      - name: Apply both reloads now
        ansible.builtin.meta: flush_handlers

    rescue:
      - name: Revert to the configuration revision recorded before the change
        ansibleguy.opnsense.raw:
          module: core
          controller: backup
          command: revertBackup
          parameters: ["{{ pre_change_revision }}"]
          action: post

      - name: Re-fail so the caller sees the failure
        ansible.builtin.fail:
          msg: "Change-set failed and was reverted to {{ pre_change_revision }}"

Two things make this a rollback rather than a stop. The revision was recorded before the block began — GET /api/core/backup/backups/this returns the history newest first, and the first entry is the state to return to. And the rescue re-fails after reverting, so the run reports failure rather than quietly succeeding on a reverted host.

The raw module is doing the revert because the collection has no module for the backup controller. That is exactly what raw is for: it takes the module, controller and command as parameters and issues the API call, with none of the idempotency the other modules provide — which is acceptable here because reverting to a named revision is a one-shot action, not a desired state.

Roles and task files

A playbook that defines five roles, each implementing a firewall area (rules, aliases, NAT, interfaces, monitoring), is the production shape. A role is a directory of tasks, defaults, handlers, and templates; the playbook imports it:

- name: Change — apply all configuration areas
  hosts: firewalls
  connection: local
  gather_facts: false
  roles:
    - role: fw_outbound_rules
      vars:
        rule_ticket: CHG-2026-1314
    - role: fw_security_aliases
      vars:
        alias_name: blocked_ips
    - role: fw_monitoring

The role directory:

roles/
  fw_outbound_rules/
    tasks/main.yml
    defaults/main.yml
    handlers/main.yml
  fw_security_aliases/
    tasks/main.yml

A role’s tasks/main.yml holds the ansibleguy.opnsense.* tasks, with defaults from defaults/main.yml that the playbook overrides per role with vars:. Role names are plain identifiers — dots in a role name are not namespacing and will confuse Ansible’s collection resolution. The discipline is one role per configuration area, owned by one team.

Serial batches across a fleet

A multi-firewall playbook uses serial to limit concurrency:

- name: apply rule group to all firewalls
  hosts: firewalls
  serial: 1
  ...

The serial: 1 runs the play against one firewall at a time, in inventory order. A failure on the first host halts the play; the operator verifies manually before continuing. The argument at the top of the play (ansible-playbook --limit fw-canary-01) runs only the canary first; a subsequent run uses --limit edge --serial 2 to run two firewalls at a time in batches.

The pattern with max_fail_percentage:

- hosts: firewalls
  serial: 5
  max_fail_percentage: 20

Up to 20% of a batch of 5 may fail (one firewall) before the play halts. The discipline: pick a percentage that catches the “most things are broken, stop before more are broken” case but tolerates a single bad host.

Worked walkthrough: a complete playbook

The canonical playbook for “add a rule across the estate, safely”:

1. reach.yml — confirm every firewall is reachable over the API. Run as the first step.
2. change-firewalls.yml — the change playbook with a single role. Block: stage the writes, flush handlers. Rescue: revert to the recorded configuration revision.
3. verify-firewalls.yml — re-run the change playbook in --check mode. Expected: zero changes (the desired state is already in place).

Three files. Three commands:

ansible-playbook -i inv/prod.yml reach.yml --ask-vault-pass
ansible-playbook -i inv/prod.yml change-firewalls.yml --serial 1 --ask-vault-pass
ansible-playbook -i inv/prod.yml change-firewalls.yml --check --ask-vault-pass

The final check-mode run should report zero changes everywhere. If any host reports changed, the previous change did not apply, and the operator investigates before declaring success.

Summary

• The four phases — reach, pre-flight, change, verify — catch different classes of failure and form a production-discipline playbook.
• Every write module applies by default. Set reload: false and reload once from a handler; flush_handlers anchors that reload at a chosen point. The block/rescue structure is the rollback path.
• Plays set connection: local and put the connection parameters in module_defaults under group/ansibleguy.opnsense.all. Roles give reuse: one role per configuration area, owned by one team.
• serial and max_fail_percentage bound the blast radius. A failed host is investigated manually before re-running.
• The playbook shape that ships in production: reach → change (with rescue) → verify (re-run for zero changes). Three files, three commands, one disciplined shape.