When the inventory source fails

A static inventory fails in one way: the file is malformed and you find out immediately. A dynamic inventory depends on somebody else’s service over somebody else’s network, which gives it four failure modes — and they are not equally easy to notice.

The two at the bottom are the expensive ones, and the last one is the subject of this lesson’s central claim:

A partial inventory is more dangerous than a missing one, because a missing one does nothing and a partial one does the wrong amount of something, successfully.

The default is a warning, and that is a deliberate design

Point Ansible at a source it cannot parse, alongside one it can:

$ ansible-inventory -i broken.yml -i static.yml --graph

[WARNING]: Failed to parse inventory with 'auto' plugin: YAML parsing failed:
While parsing a flow node did not find expected node content.
[WARNING]: Failed to parse inventory with 'yaml' plugin: YAML parsing failed:
While parsing a flow node did not find expected node content.
[WARNING]: Failed to parse inventory with 'ini' plugin: Failed to parse inventory:
Expected key=value host variable assignment, got: is
[WARNING]: Unable to parse .../broken.yml as an inventory source
@all:
|--@ungrouped:
|--@do_not_automate:
|  |--db01.example.com

Exit code 0.

That is not a bug. A directory inventory is expected to contain files that most plugins cannot parse — an INI file that the YAML plugin refuses, a plugin config that the INI plugin refuses — so “cannot parse” has to be survivable or mixed inventories would be impossible.

The consequence, however, is that a genuinely broken source and a perfectly normal directory look the same from the exit code.

Two settings that change it

Ansible ships two configuration options for this, and they are not the same. Both are false by default.

$ ansible-config list | sed -n '/^INVENTORY_ANY_UNPARSED_IS_FAILED:/,/^INVENTORY_ENABLED:/p'

INVENTORY_ANY_UNPARSED_IS_FAILED:
default: false
description: 'If ''true'', it is a fatal error when any given inventory source cannot
  be successfully parsed by any available inventory plugin; otherwise, this situation
  only attracts a warning.
  '
env:
- name: ANSIBLE_INVENTORY_ANY_UNPARSED_IS_FAILED
ini:
- key: any_unparsed_is_failed
  section: inventory
name: Controls whether any unparsable inventory source is a fatal error
type: boolean
version_added: '2.7'

The difference decides whether you catch the interesting case. With a mixed inventory, one broken dynamic source and one working static file means “every source” is false — so the first setting does nothing:

$ ANSIBLE_INVENTORY_UNPARSED_FAILED=true ansible-inventory -i broken.yml -i static.yml --graph; echo exit=$?

exit=0

$ ANSIBLE_INVENTORY_ANY_UNPARSED_IS_FAILED=true ansible-inventory -i broken.yml -i static.yml --graph; echo exit=$?

[ERROR]: Completely failed to parse inventory source .../broken.yml
exit=1

The failure Ansible cannot detect

Now the one no setting fixes.

An API call succeeds and returns fewer resources than exist. There are several ordinary ways this happens, and none of them is an error:

• Rate limiting. The provider returns 429 for some pages; the SDK retries, gives up, and the plugin gets what it got.
• Pagination cut short. A token expires mid-enumeration, or a page boundary is mishandled under load.
• Eventual consistency. A regional endpoint has not yet caught up with instances created minutes ago.
• Narrowed permissions. A credential lost access to one project. Everything it can see, it returns, correctly.

In all four the HTTP status is 200 and the plugin behaves perfectly. It reports what it was told. Ansible has no basis on which to doubt it — there is no expected count anywhere in the system, which is the same structural fact behind everything else in this part.

Why partial beats missing, for damage

Compare the two directly on a patching run:

The empty case is loud enough that somebody eventually asks why the recap is blank. The partial case produces a report that says “142 hosts patched successfully”, which is exactly what a healthy run says. Nobody asks a question about a green run.

And on a rolling deployment the partial case is worse than “behind”: you deploy a new version to the hosts the API returned, the load balancer keeps sending traffic to the ones it did not, and you now have two versions serving simultaneously with no record of which is where.

Making zero hosts fail

The most common concrete version of all this is a run that matches nothing. Ansible’s default is unambiguous and unhelpful:

$ ansible-playbook -i inventory/ nohosts.yml; echo exit=$?

[WARNING]: Could not match supplied host pattern, ignoring: nosuchgroup

PLAY [Target a group that does not exist] **************************************
skipping: no hosts matched

PLAY RECAP *********************************************************************

exit=0

For interactive use that is reasonable behaviour. For automation it is not, because “the job succeeded” and “the job did nothing” have become the same observable event.

Two layers fix it, and they catch different things.

Layer one: a pre-flight play that asserts a floor. It runs before the change and refuses to continue on an implausible host list.

- name: Pre-flight - the inventory must be plausible
hosts: linux_servers
gather_facts: false
vars:
  expected_min_hosts: 100
tasks:
  - name: Refuse to proceed on an implausible host list
    run_once: true
    ansible.builtin.assert:
      that:
        - ansible_play_hosts_all | length >= expected_min_hosts | int
      fail_msg: >-
        Inventory returned {{ ansible_play_hosts_all | length }} hosts,
        expected at least {{ expected_min_hosts }}. Treating this as a
        failed inventory query rather than a small fleet.
      success_msg: "{{ ansible_play_hosts_all | length }} hosts, at or above the floor"

That catches the partial result and the empty one, because both produce a count below the floor. It does not catch zero hosts, for a reason worth understanding: with no hosts in the play, there is nothing for the task to run on, so the assertion never executes.

Layer two: the wrapper, which is what catches zero. The host count has to be checked from outside the play:

#!/usr/bin/env bash
set -euo pipefail

PATTERN=${1:?usage: run-change.sh <pattern>}
FLOOR=${FLOOR:-1}

count=$(ansible -i inventory/ "$PATTERN" --list-hosts       | sed -n 's/^ *hosts (\([0-9]*\)):.*/\1/p')

if [ -z "$count" ] || [ "$count" -lt "$FLOOR" ]; then
echo "refusing: pattern '$PATTERN' resolved to ${count:-0} hosts, floor is $FLOOR" >&2
exit 1
fi

echo "$PATTERN resolves to $count hosts; proceeding"
ansible-playbook -i inventory/ site.yml --limit "$PATTERN"

$ run-change.sh nosuchgroup; FLOOR=20 run-change.sh web

[WARNING]: Could not match supplied host pattern, ignoring: nosuchgroup
[WARNING]: No hosts matched, nothing to do
refusing: pattern 'nosuchgroup' resolved to 0 hosts, floor is 1
rc=1

refusing: pattern 'web' resolved to 4 hosts, floor is 20
rc=1

1. Set any_unparsed_is_failed = true so a source that cannot be read stops the run instead of shrinking it.
2. Snapshot the host count for every pattern a scheduled job targets, and compare on each run.
3. Assert a floor inside the play, sized against the snapshot rather than guessed.
4. Wrap scheduled runs so a zero-host resolution exits non-zero before the playbook is invoked.
5. Alert on a run whose changed count is zero when it has never previously been zero - the honest end state, where the automation reports what it did and something outside it decides whether that is plausible.

Knowledge check