Proving which hosts you are about to touch

You have already met the inventory tools as a pre-flight discipline — print what hosts: all means before you rely on it. This lesson uses the same tools for a different job: diagnosis, when the host list is not what you expected and you need to find out why.

Two questions come up over and over, and neither is answerable by reading the inventory files:

• Why is this host in this group?
• Why is this host not in the group I expected?

The first is more common in incidents. A change reached a host it should not have, and someone needs to explain how.

The four views, and what each is for

$ ansible-inventory -i inv.ini --graph

@all:
|--@ungrouped:
|--@local:
|  |--lh
|--@far:
|  |--web01
|  |--web02

For debugging, --graph --vars and --host do most of the work, and --yaml is the one to use when you want to compare two machines.

Why is this host in this group?

Membership has exactly four possible sources, and the diagnosis is a matter of eliminating them in order.

1. It is listed explicitly. Grep the inventory tree for the hostname. This is the case people check, and it is the least common cause of a surprise.

2. It is a member through a children relationship. A host in web_prod is also in webservers if webservers has web_prod as a child. --graph shows this as nesting; a flat mental model of the inventory does not.

3. A pattern or range expanded to it. An entry like web[01:50].example.com in an INI inventory produces fifty hosts, and the hostname you are looking for is in there without appearing literally.

4. A dynamic source or constructed plugin created it. A cloud inventory plugin invents groups from tags; the constructed plugin creates groups from Jinja expressions over facts and variables. Neither appears in any file that mentions the group name next to the host name.

h=app-047.example.com

# 1. Listed literally anywhere?
grep -rn "$h" inventory/ || echo 'not literal'

# 2. Which groups does the resolved inventory place it in?
ansible-inventory -i inventory/prod --host "$h" | grep -A 20 group_names \
|| ansible -i inventory/prod "$h" -m debug -a 'var=group_names'

# 3. Which group definitions expand to ranges?
grep -rnE '\[[0-9]+:[0-9]+\]' inventory/

# 4. Which inventory sources are in play at all?
ansible-inventory -i inventory/prod --list | head -5
ansible-config dump --only-changed | grep -i inventory

Why is this host not in the group?

The mirror-image question, and it fails silently in a way the first one does not: a pattern that matches nothing produces a warning and an empty run, and an empty run looks like a fast successful run.

The three usual causes:

A name mismatch. The inventory knows app-047 and you typed app047, or the inventory knows the FQDN and the group definition uses the short name. Inventory hostnames are strings; nothing normalises them.

Group membership defined in the wrong direction. children goes from parent to child. Adding webservers as a child of web_prod when you meant the reverse produces a tree that looks plausible in a diff and places hosts in the wrong place.

Source order. When two inventory sources define the same group, the result depends on how they merge, and a host defined in a file that is not being read at all is the commonest cause of “it is right there in the inventory”.

$ ansible-playbook -i inv.ini one.yml -vvv 2>&1 | grep -i 'inventory source'

Parsed /home/ops/estate/inv.ini inventory source with ini plugin

That single line resolves more inventory arguments than any amount of reading files. If the file you have been editing is not in that list, nothing else you check matters.

The pattern is the blast radius

--list-hosts is the gate. It resolves the same pattern the run will use, through the same inventory, and prints the result without connecting to anything.

$ ansible-playbook -i inventory/prod patch.yml --limit @retarget.txt --list-hosts

playbook: patch.yml

play #1 (appservers): Patch and configure    TAGS: []
  pattern: ['appservers']
  hosts (80):
    app-047.example.com
    app-112.example.com
    ...

Read the count first. hosts (80) against a limit file you know contains 80 lines is a pass. hosts (77) means three names in your file do not exist in the inventory, and Ansible will proceed anyway.

Diagnosing “it works on my machine”

When two people get different host lists from the same repository, the difference is in one of four places, and --yaml finds it in a minute.

# on each machine, from the same commit
ansible-inventory -i inventory/prod --list --yaml > inv-$(hostname -s).yaml

# then compare
diff -u inv-controller.yaml inv-laptop.yaml | head -40

# and check the four usual suspects
ansible --version | grep -E 'core|config file'
ansible-config dump --only-changed | grep -iE 'inventory|vault|collections'
git -C . rev-parse --short HEAD
ls -l inventory/

The four suspects, in order of how often they are the answer: a different ansible.cfg in effect, a different working directory changing which relative inventory path resolves, a different commit checked out, and an environment variable (ANSIBLE_INVENTORY) set in one shell and not the other.

Knowledge check