Runbook: Configure a new inventory

The inventory is the blast-radius map. Every hosts: line, every --limit, every group pattern resolves through it, and none of them can be safer than the file they resolve against. A new inventory is therefore not a data-entry task - it is a change to what your automation is capable of touching.

This runbook brings one into service and proves it before anything runs against it in anger.

When to use this runbook

• A new environment, region or tenant is being brought under management.
• An existing static inventory is being restructured or split.
• A team is taking over hosts that were previously managed elsewhere.
• A dynamic inventory source is being replaced by, or supplemented with, a static one.

Blast radius

Writing the inventory: none. Steps 1 to 6 are file edits and read-only resolution.

Step 7 connects to every host in the new file. Step 8 runs a playbook against one of them in check mode. The number that matters throughout is the host count in Step 3 - if that number is larger than you expected, every later step is larger than you expected too.

Inputs

• The authoritative list of hosts, exported from the source of truth (CMDB, cloud API, provisioning system) with a timestamp.
• The environment name and the repository’s convention for it.
• The connection details: user, port, bastion, key.
• The group structure this estate uses for roles and for lifecycle.

Step 1: Create the layout

mkdir -p inventories/staging/group_vars/all
mkdir -p inventories/staging/host_vars
touch inventories/staging/hosts.yml

One directory per environment, and the environment name in the path. This is what makes --limit mistakes survivable: a play run against inventories/staging cannot reach production hosts, because they are not in the file.

Step 2: Write the hosts and groups

all:
children:
  web:
    hosts:
      web01.example.com:
        ansible_host: 192.0.2.11
      web02.example.com:
        ansible_host: 192.0.2.12
  db:
    hosts:
      db01.example.com:
        ansible_host: 192.0.2.21
  staging:
    children:
      web:
      db:

Two kinds of group, and it is worth keeping them visibly separate:

• Function groups (web, db, cache) say what a host is. Roles target these.
• Lifecycle groups (staging, canary, wave1) say how a host is treated. Rollouts target these.

A host belongs to exactly one function group and to as many lifecycle groups as the process needs. Mixing the two - a group called web-staging-wave1 - produces a combinatorial explosion that nobody maintains, and the group that is missing on the day you need it is the one you have to invent under pressure.

Step 3: Render the graph and read it

ansible-inventory -i inventories/staging --graph

Verified output shape on 2.21.3:

@all:
  |--@ungrouped:
  |--@web:
  |  |--web01.example.com
  |  |--web02.example.com
  |--@db:
  |  |--db01.example.com

Three things to read, in this order:

1. @ungrouped must be empty. A host that landed there matches no function group, so no role targets it, and it will silently receive nothing while appearing to be managed.
2. Every group you expected exists, and no group you did not expect does. A group that appears from nowhere usually means an indentation error promoted a host list into a child group.
3. The host count. Count it and compare against the source of truth.

ansible-inventory -i inventories/staging --list \
| python3 -c 'import json,sys; print(len(json.load(sys.stdin)["_meta"]["hostvars"]))'

If that number does not match the export you started from, do not proceed. An inventory that is missing hosts fails quietly - those hosts just never get patched. An inventory with extra hosts fails loudly, at 3am, on a machine that belongs to another team.

Step 4: Resolve variables and check where each came from

ansible-inventory -i inventories/staging --host web01.example.com

Sample one host per group. For each variable that matters, ask which layer it came from and whether that was deliberate.

Step 5: Look for conflicting group membership

A host in two groups that both set the same variable gets one of them, and which one depends on group depth and on alphabetical order of group names at the same depth. That is a rule, but it is not a rule anyone should be relying on in production.

# Which groups is each host in?
ansible-inventory -i inventories/staging --graph --vars

# Which groups define the same key?
grep -rn 'app_port' inventories/staging/group_vars/

If two group_vars files set the same key and a host is in both, either move the value to one layer or make the intent explicit with a host_vars override. Leaving it to resolution order is how a value changes because someone renamed a group.

Step 6: Prove what a play would target

ansible-playbook -i inventories/staging site.yml --list-hosts
ansible-playbook -i inventories/staging site.yml --limit web --list-hosts
ansible-playbook -i inventories/staging patch.yml --list-tasks

Do this for every playbook that will use this inventory, not just the first one. Each play has its own hosts: line, and a play whose pattern does not match anything in the new inventory is a play that will be skipped silently on the day it matters.

Verified on 2.21.3: a --limit that matches nothing does not silently run against everything. It produces

[WARNING]: Could not match supplied host pattern, ignoring: nosuchhost*
[ERROR]: Specified inventory, host pattern and/or --limit leaves us with no hosts to target.

and exit code 1. That is the desired behaviour and it is worth knowing it is the behaviour, because it means a mistyped limit stops rather than widens.

Step 7: Prove reachability

ansible -i inventories/staging all -m ping -o

ping here is Ansible’s module, not ICMP: it opens the connection, runs a tiny module on the target and returns pong. That is the point - it proves transport, authentication and the remote Python at once.

Do not build this check out of debug. debug is an action plugin that runs on the controller and never connects, so a genuinely unreachable host reports unreachable=0 and the check passes on a fleet you cannot reach. Exit code 4 is what an unreachable host produces; check for it.

Step 8: One check-mode run

ansible-playbook -i inventories/staging site.yml \
--limit web01.example.com --check --diff

One host, check mode, diff on. You are testing that the inventory feeds the playbook sensible values, so read the diff for variables that rendered wrong - an empty string where a hostname should be, a default that should have been overridden - rather than for whether the change itself is correct.

Step 9: Commit and reconcile

Commit the inventory with the source-of-truth export referenced in the message, and record in the change record:

• The export timestamp and the host count it contained.
• The rendered host count from Step 3.
• Any difference, with a reason.
• Which playbooks were --list-hosts checked against it.

A reconciliation that was done and not written down has to be done again by the next person, who will not know it already agreed.

Rollback

The inventory is a file. Before merge, delete the directory. After merge, git revert the commit.

git revert --no-edit <inventory-commit>
ansible-inventory -i inventories/staging --graph

Point of no return: the revert is only a clean rollback while the inventory has never been used for a run that changed something. Once a play has executed against it, reverting the file changes nothing on the hosts. If the inventory targeted hosts it should not have, the file revert is step zero and the configuration rollback runbook is the actual recovery.

Common patterns

Escalation

Escalate when:

• The rendered count and the source of truth disagree and you cannot explain the difference. An inventory you cannot reconcile is an inventory whose blast radius you cannot state.
• A host would be present in two environment inventories.
• The convention here would have to differ from the rest of the repository. Convention drift compounds; get it decided once.
• Reachability fails at group granularity. That is infrastructure, and the inventory change should wait behind it.