Building and proving a dynamic source

There is a version of this lesson that consists of a working cloud plugin config you paste in. It would be shorter and it would teach you nothing, because the difficult part of a dynamic source is never the YAML. It is knowing, before you run a change, that the host list in front of you is the host list the source will produce tomorrow.

So the shape of this lesson is a bring-up order. Five steps, each of which produces evidence, and none of which involves a playbook. A playbook enters the picture only at the end, in check mode, against one host.

The example is deliberately not a cloud

Every provider plugin — amazon.aws.aws_ec2, community.general.proxmox, azure.azcollection.azure_rm, openstack.cloud.openstack — has the same three-part shape: authentication, a query, and a mapping from provider metadata to Ansible groups and variables. The vendor documentation covers the first two well and the third badly.

ansible.builtin.constructed has the third part and none of the first two. It takes hosts that already exist and derives groups and variables from their data. That makes it the honest teaching vehicle: you can run every command here, get identical output, and not need an account anywhere.

all:
hosts:
  web01.example.com:
    provider_tags: {role: web, env: prod, region: eu-west-1}
  web02.example.com:
    provider_tags: {role: web, env: prod, region: us-east-1}
  db01.example.com:
    provider_tags: {role: db, env: prod, region: eu-west-1}
  stage-web01.example.com:
    provider_tags: {role: web, env: stage, region: eu-west-1}

plugin: ansible.builtin.constructed
strict: true
keyed_groups:
- key: provider_tags.role
  prefix: role
  separator: '_'
- key: provider_tags.env
  prefix: ''
  separator: ''
- key: provider_tags.region
  prefix: region
  separator: '_'
groups:
needs_patching: provider_tags.env == 'prod'
compose:
patch_window: (provider_tags.env == 'prod') | ternary('sunday-0200', 'anytime')

A real cloud config differs in the top third and is identical in the bottom two-thirds. Where this one reads provider_tags.role, an EC2 config reads tags.Role; where this one has no credentials, that one has a region list and a profile. The keyed_groups, groups and compose keys are the same keys, because they come from a shared implementation.

Step one: did the plugin load at all?

Not “are the hosts right” — that is step two. This step asks only whether the source was parsed.

$ ansible-inventory -i inventory/ --graph

@all:
|--@ungrouped:
|--@needs_patching:
|  |--web01.example.com
|  |--web02.example.com
|  |--db01.example.com
|--@role_web:
|  |--web01.example.com
|  |--web02.example.com
|  |--stage-web01.example.com
|--@prod:
|  |--web01.example.com
|  |--web02.example.com
|  |--db01.example.com
|--@region_eu_west_1:
|  |--web01.example.com
|  |--db01.example.com
|  |--stage-web01.example.com
|--@region_us_east_1:
|  |--web02.example.com
|--@role_db:
|  |--db01.example.com
|--@stage:
|  |--stage-web01.example.com

The test is not “did I get hosts”. You would get hosts from the static file alone. The test is did I get something only the plugin could have produced — a derived group, a composed variable, a host that exists nowhere in a file.

Step two: what exactly does it return?

--graph answers structure. --list answers everything, and --yaml makes it readable rather than diffable.

$ ansible-inventory -i inventory/ --list --yaml

all:
children:
  needs_patching:
    hosts:
      db01.example.com:
        patch_window: sunday-0200
        provider_tags:
          env: prod
          region: eu-west-1
          role: db
      web01.example.com:
        patch_window: sunday-0200
        provider_tags:
          env: prod
          region: eu-west-1
          role: web
  prod:
    hosts:
      db01.example.com: {}
      web01.example.com: {}
      web02.example.com: {}
  stage:
    hosts:
      stage-web01.example.com: {}

For a single host, --host gives a clean object that is easy to eyeball and easy to diff:

$ ansible-inventory -i inventory/ --host stage-web01.example.com

{
  "patch_window": "anytime",
  "provider_tags": {
      "env": "stage",
      "region": "eu-west-1",
      "role": "web"
  }
}

Two hosts are worth checking by hand every time, and they are never the obvious ones:

• The host you expect to be excluded. If stage-web01 should not be in needs_patching, confirm it is not, rather than confirming that web01 is.
• The host with incomplete metadata. Every real estate has one machine somebody created without the tags. It is the host your config was not written for, and it is the host that will behave unexpectedly.

The bug this catches, which review does not

Here is a compose line that looks correct and is wrong:

compose:
patch_window: provider_tags.env == 'prod' | ternary('sunday-0200', 'anytime')

Jinja applies the filter before the comparison, so the expression is provider_tags.env == ('prod' | ternary(...)). The result is not an error. It is false, on every host:

$ ansible-inventory -i inventory/ --host web01.example.com

{
  "patch_window": false,
  "provider_tags": {
      "env": "prod",
      "region": "eu-west-1",
      "role": "web"
  }
}

The fix is a pair of brackets: (provider_tags.env == 'prod') | ternary(...). The point is not the bracket. The point is that this class of defect is invisible in review, invisible in the exit code, and takes four seconds to find with --host. Any downstream when: patch_window == 'sunday-0200' would silently match nothing, forever.

Step three: prove the pattern, not just the inventory

The groups exist. The next question is whether the group names you invented are the group names you would type under pressure.

$ ansible -i inventory/ 'needs_patching:&region_eu_west_1' --list-hosts

  hosts (2):
  web01.example.com
  db01.example.com

Read the count. Two, not four — the region intersection did its job. Part XXX is entirely about this habit; here it is the acceptance test for the group design.

The bring-up order

1. Install the collection and read the plugin documentation: ansible-galaxy collection install <namespace.name>, then ansible-doc -t inventory <fqcn>. Read the options list before writing the config, not after it fails.
2. Write the config with the narrowest query the plugin supports - one region, one project, one tag - and point -i at that file alone. A first source that returns forty hosts is easier to check than one that returns four thousand.
3. Prove the plugin ran: ansible-inventory --graph, and look for something only the plugin could have produced.
4. Prove the contents: ansible-inventory --list --yaml for the whole picture, then --host on the host you are least confident about and on the host with missing metadata.
5. Prove the pattern: ansible <pattern> --list-hosts, and read the count against what you expect.
6. Only now involve a playbook, and only in check mode, and only against one host: ansible-playbook site.yml --limit <one-host> --check --diff.

Step two is where people overreach. A source scoped to everything is a source you cannot verify by reading, and the whole value of this sequence is that a human confirms the output at least once before it becomes the thing that decides which machines get changed.

Knowledge check