Preconditions on capacity and health · Ansible

The guards so far have asserted on things the controller knows for certain: which inventory this is, which groups the host is in, how many hosts the play resolved to. Those facts cannot be stale, because they were read from disk a second ago.

Capacity and health guards are different, and the difference is the whole lesson. free disk > 20 GB is not a fact about the world. It is a fact about some measurement of the world, taken at some time, by some mechanism. The assertion is worth exactly as much as that measurement, and a guard nobody has questioned on those terms is decoration.

The four checks worth building

Check	The change it protects	Where the evidence comes from
Capacity	Anything that writes: a package upgrade, a log-heavy migration, a database import	Gathered facts (`ansible_facts.mounts`)
Service health	Anything that restarts a service, or takes a node out of rotation	A live probe of the service or its health endpoint
Cluster quorum	Draining, rebooting or removing a node from a cluster	The cluster API, queried once from the controller
Backup freshness	Anything irreversible	The backup system, not the playbook’s belief about it

Capacity, from gathered facts

ansible_facts.mounts is a list of dictionaries, one per mount, each carrying size_total and size_available in bytes.

- name: Refuse if the root filesystem is too full for the upgrade
  ansible.builtin.assert:
    that:
      - root_free_bytes | int > (required_gb | int * 1024 * 1024 * 1024)
    fail_msg: >-
      Refusing: / on {{ inventory_hostname }} has
      {{ (root_free_bytes | int / 1024 / 1024 / 1024) | round(1) }} GB
      free and this upgrade needs {{ required_gb }} GB. Clear space or
      exclude this host with --limit.
    success_msg: 'Capacity precondition met on /.'
  vars:
    root_free_bytes: >-
      {{ ansible_facts.mounts | selectattr('mount', 'equalto', '/')
         | map(attribute='size_available') | first }}

Service impact possiblea capacity guard refusing— Verified on ansible-core 2.21.3 against a real filesystem. Facts were gathered in the same run, so the measurement is seconds old. Severity reflects the play, not this run: had the guard passed, the upgrade would have restarted services.

$ ansible-playbook -i inv.ini upgrade.yml

TASK [Refuse if the root filesystem is too full for the upgrade] ***************
fatal: [localhost]: FAILED! => {
  "assertion": "root_free_bytes | int > (required_gb | int * 1024 * 1024 * 1024)",
  "changed": false,
  "evaluated_to": false,
  "msg": "Refusing: / has 63.9 GB free and this upgrade needs 488.3 GB. Clear space or exclude this host with --limit."
}

The message reports the observed value, not just the rule. An operator who sees 63.9 GB free, 488.3 GB needed knows immediately whether to clear space or to question the requirement; an operator who sees insufficient disk space has to go and measure it themselves.

Guarding `/` when the write lands on `/var`

selectattr('mount', 'equalto', '/') measures the root filesystem. If /var is a separate filesystem — which it is on most machines that matter — then the guard measured a filesystem the change will not touch, and passed.

Assert on the mount that receives the write. Where that depends on the host’s layout, resolve it from the facts rather than assuming:

- name: Find the filesystem that will receive the write
  ansible.builtin.set_fact:
    target_mount: >-
      {{ ansible_facts.mounts
         | selectattr('mount', 'in', ['/var/lib/postgresql', '/var', '/'])
         | sort(attribute='mount') | last }}

The general failure is measuring something adjacent to what matters, and it is the commonest way a capacity guard is technically correct and operationally useless.

The question that makes a guard real

For every precondition, two questions:

Where did this fact come from? A gathered fact, a live probe, a registered result from earlier in the play, an external system, or a variable somebody typed into group_vars in 2024.

How old is it? Seconds, or days.

The second question has a specific and unpleasant answer whenever fact caching is enabled.

Read-only / Safethe cache settings that decide the answer— Verified on ansible-core 2.21.3. Reads configuration; changes nothing. The defaults shown are for a controller with no ansible.cfg.

$ ansible-config dump | grep ^CACHE_PLUGIN

CACHE_PLUGIN(default) = memory
CACHE_PLUGIN_CONNECTION(default) = None
CACHE_PLUGIN_PREFIX(default) = ansible_facts
CACHE_PLUGIN_TIMEOUT(default) = 86400

The names in ansible.cfg differ from the internal names, which is worth knowing when you go looking:

[defaults]
fact_caching = jsonfile
fact_caching_connection = /var/cache/ansible/facts
fact_caching_timeout = 86400

CACHE_PLUGIN defaults to memory, which is per-run and therefore always fresh. The moment somebody sets fact_caching to jsonfile or redis — usually for a good reason, because gathering facts on three thousand hosts is slow — every gathered fact in every guard becomes up to fact_caching_timeout seconds old. The default is 86400: one full day.

A disk guard reading a cached fact is a guard against yesterday

The incident writes itself. Fact caching is enabled with the default timeout. A play runs with gather_facts: false to save time, relying on cached facts. The capacity guard reads size_available from a cache written three days ago, when the filesystem had plenty of room. The assertion passes. The change runs. The filesystem fills. The service stops.

Every component behaved as documented. The guard was correct. The measurement was three days old, and nothing in the playbook, the output or the recap said so.

Two defences, and you want both:

Gather the facts the guard depends on, in the run that uses them. A targeted gather is cheap:

- name: Refresh only the facts the capacity guard needs
  ansible.builtin.setup:
    gather_subset:
      - '!all'
      - '!min'
      - hardware
  check_mode: false

Assert on freshness, not only on value. If the guard cannot refresh the fact, it must at least refuse to trust an old one — which requires the fact’s timestamp to exist, and is the reason the next section exists.

Freshness as its own assertion

Backup freshness is the clearest case, because the value being checked is a time.

- name: Refuse if the last backup is older than this change is reversible
  ansible.builtin.assert:
    that:
      - backup_age_hours | float < max_backup_age_hours | float
    fail_msg: >-
      Refusing: the last successful backup of {{ inventory_hostname }}
      completed {{ backup_age_hours }} hours ago
      ({{ backup_completed_at }}) and this change is not reversible.
      Take a backup first, or set -e skip_backup_check=true and record
      why in the change ticket.
    success_msg: >-
      Backup precondition met: last backup {{ backup_age_hours }} hours
      old, limit {{ max_backup_age_hours }}.
  vars:
    backup_age_hours: >-
      {{ (((ansible_facts.date_time.epoch | int) -
           ((backup_completed_at | to_datetime('%Y-%m-%dT%H:%M:%SZ')).strftime('%s') | int))
          / 3600) | round(1) }}

Service impact possiblea stale backup refusing the change— Verified on ansible-core 2.21.3. backup_completed_at was 2026-08-08T02:14:00Z; the run was on 2026-08-11. Severity reflects what the play does once the guard passes - this is the irreversible change the guard exists for.

$ ansible-playbook -i inv.ini migrate.yml

TASK [Refuse if the last backup is older than this change is reversible] *******
fatal: [localhost]: FAILED! => {
  "assertion": "backup_age_hours | float < max_backup_age_hours | float",
  "changed": false,
  "evaluated_to": false,
  "msg": "Refusing: the last successful backup of localhost completed 91.6 hours ago (2026-08-08T02:14:00Z) and this change is not reversible. Take a backup first, or set -e skip_backup_check=true and record why in the change ticket."
}

The critical design point is where backup_completed_at comes from. If it is a variable in group_vars, the guard is theatre — it checks a number a human wrote down. It is only a guard if the value is queried from the backup system in this run:

- name: Ask the backup system when this host was last backed up
  ansible.builtin.uri:
    url: 'https://backups.example.com/api/v1/hosts/{{ inventory_hostname }}/latest'
    headers:
      Authorization: 'Bearer {{ backup_api_token }}'
    return_content: true
  register: backup_status
  delegate_to: localhost
  check_mode: false
  no_log: true

delegate_to: localhost because the controller has the credential and the target does not. check_mode: false because this is a read-only query that must still run during a --check — the check-mode part explains why that keyword is load-bearing here. no_log: true because the URL carries a bearer token.

Health and quorum: ask once, not once per host

A cluster has one health state. Querying it two hundred times is two hundred chances to be rate-limited and one answer.

- name: Read cluster health once, from the controller
  ansible.builtin.uri:
    url: 'https://cluster.example.com/api/health'
    return_content: true
  register: cluster
  run_once: true
  delegate_to: localhost
  check_mode: false

- name: Refuse to drain a node from a cluster that is already degraded
  ansible.builtin.assert:
    that:
      - cluster.json.status == 'healthy'
      - cluster.json.nodes_unavailable | int == 0
    fail_msg: >-
      Refusing to drain {{ inventory_hostname }}: the cluster reports
      '{{ cluster.json.status }}' with
      {{ cluster.json.nodes_unavailable }} nodes already unavailable.
      Draining another node risks quorum.

The assert deliberately is not run_once. The probe is a fact about the cluster and is read once; the decision is about this host and is made per host — so a serial rollout re-evaluates it for every batch, which is exactly what you want when the cluster degrades halfway through.

Gathering less, and where fact names are going

gather_subset controls how much the setup module collects, and it is the difference between a guard that costs 200 milliseconds and one that costs two seconds per host:

- hosts: webservers
  gather_facts: true
  gather_subset:
    - '!all'
    - '!min'
    - hardware      # mounts, memory, CPU

!all and !min switch everything off, then named subsets are added back. hardware carries mounts and memtotal_mb; network carries interfaces and addresses; date_time carries the clock. On a fleet where fact gathering dominates run time, a play that needs only mounts should say so.

Two deprecations from 2.21.3 are worth writing new guards around rather than retrofitting later. Both appear as warnings on any run that uses the old form:

[DEPRECATION WARNING]: INJECT_FACTS_AS_VARS default to `True` is deprecated,
top-level facts will not be auto injected after the change. This feature will
be removed from ansible-core version 2.24.

Written out: ansible_mounts and ansible_date_time are top-level injections of ansible_facts.mounts and ansible_facts.date_time, and the injection is going away. Every guard in this lesson uses the ansible_facts.<name> form for that reason.

The second, from the assertions lesson, applies to any condition: templating delimiters inside that, when, changed_when and failed_when are deprecated for removal in 2.23.

Neither breaks anything today. Both mean that a guard written the old way is a guard somebody has to revisit under time pressure when a controller is upgraded — and a guard nobody revisits is a guard that gets deleted.

What happens if the system the guard queries is down?

The guard fails, the change is refused, and this is correct — but it needs to be a decision rather than an accident.

A health probe against an unreachable API produces a task failure whose message is about HTTP, not about your change:

fatal: [web01.example.com]: FAILED! => {"msg": "Status code was -1 and not [200]:
  Request failed: <urlopen error [Errno 111] Connection refused>"}

The operator now has to work out whether the cluster is unhealthy or the monitoring is broken, and those have opposite correct actions.

Make the distinction explicit:

- name: Read cluster health
  ansible.builtin.uri:
    url: 'https://cluster.example.com/api/health'
    return_content: true
  register: cluster
  run_once: true
  delegate_to: localhost
  check_mode: false
  failed_when: false

- name: Refuse if cluster health could not be established
  ansible.builtin.assert:
    that:
      - cluster.status == 200
    fail_msg: >-
      Refusing: could not read cluster health from
      cluster.example.com (HTTP {{ cluster.status }}). This is not a
      statement that the cluster is unhealthy - it is a statement that
      we do not know. Establish health manually before proceeding.

“We do not know” is a legitimate and underused third answer. A guard that treats unknown as healthy is the same mistake as a default() that supplies the passing value; a guard that treats unknown as unhealthy without saying so sends an operator hunting a cluster problem that does not exist.

Knowledge check

Why this matters

Capacity and health guards are the ones that get written first and reviewed least, because they look self-evidently sensible. Check there is enough disk before the upgrade is not a claim anyone argues with.

The argument worth having is the second one: what measurement, taken when, of which filesystem, on whose authority. That conversation takes five minutes and it is the difference between a guard that stops the incident and a guard that is quoted in the post-mortem as having passed.

The habit that survives this lesson is a small one. Next to every precondition in your repository, in a comment or in the success_msg, record the source and the freshness of its evidence. If you cannot state them, you have not written a guard — you have written a hope with an assert around it.