Turning provider metadata into targetable groups

A provider hands you a flat list of machines with metadata attached: tags, labels, custom fields, a CMDB record. Ansible wants groups. The mapping between the two is the part of a dynamic inventory config you write yourself, and it is the part with operational consequences, because a group is what a --limit names and a --limit is what bounds a change.

Three keys do the work, and they appear with the same names and the same semantics in essentially every inventory plugin, because they come from a shared implementation:

keyed_groups: one group per value

You give it a key. It reads that key on every host, and creates a group named after each distinct value.

keyed_groups:
- key: tags.Role
  prefix: role
  parent_group: managed
- key: tags.Env
  prefix: env
  default_value: untagged
- key: tags['Cost Centre']
  prefix: cc

Against two hosts — one fully tagged, one with an empty Env and no cost centre — that config produces exactly this:

$ ansible-inventory -i inventory/ --graph

@all:
|--@ungrouped:
|--@managed:
|  |--@role_web:
|  |  |--web01.example.com
|  |  |--web02.example.com
|--@env_prod:
|  |--web01.example.com
|--@cc_eng_1234:
|  |--web01.example.com
|--@env_untagged:
|  |--web02.example.com

The name is assembled, not copied

This is where predictions go wrong. The group name is prefix + separator + sanitised value, and every part of that has a default you did not write:

$ ansible-doc -t inventory ansible.builtin.constructed

   keyed_groups  Add hosts to group based on the values of a variable.
      suboptions:
         default_value  The default value when the host variable's
                        value is 'None' or an empty string.
                        This option is mutually exclusive with
                        'keyed_groups[].trailing_separator'.
        default: null
         key  The key from input dictionary used to generate groups.
         parent_group  parent group for keyed group.
         prefix  A keyed group name will start with this prefix.
        default: ''
         separator  separator used to build the keyed group name.
        default: _
         trailing_separator  Set this option to 'false' to omit the
                             'keyed_groups[].separator' after the
                             host variable when the value is 'None'
                             or an empty string.

Two consequences worth committing to memory.

Characters that are not legal in a group name become underscores. eng-1234 becomes eng_1234; eu-west-1 becomes eu_west_1; web-01.prod becomes web_01_prod. A dot and a hyphen are both perfectly normal in a cloud tag and neither survives.

With no prefix, the default separator still applies, and you get a leading underscore. A keyed_groups entry with only a key produces _prod, not prod:

$ ansible-inventory -i inventory/ --graph

@all:
|--@ungrouped:
|  |--untagged01.example.com
|--@_prod:
|  |--web01.example.com

# after adding  leading_separator: false

@all:
|--@ungrouped:
|  |--untagged01.example.com
|--@prod:
|  |--web01.example.com

leading_separator is a top-level option, not a suboption of a single keyed_groups entry. It applies to every entry that has no prefix.

Missing and empty values

Two different situations, handled differently.

The key is absent — the host has no Cost Centre tag at all. The host joins no cc_ group. This is usually what you want, and it is the default with strict: false.

The key is present but empty or null. Without help you get a group whose name is just the prefix and separator — env_ — which is a real group with a strange name that will confuse the next person. Two mutually exclusive options fix it:

• default_value: untagged substitutes a value, giving env_untagged.
• trailing_separator: false drops the separator, giving env.

default_value is almost always the better choice, because env_untagged is a group you can deliberately exclude and env is a group whose name tells you nothing.

groups: a condition, not a value

keyed_groups cannot express “production machines in Europe that are not databases”. groups can, because it takes a Jinja conditional per group name:

groups:
needs_patching: provider_tags.env == 'prod'
eu_data_residency: provider_tags.region is match('eu-.*')
automation_opt_in: provider_tags.get('ansible_managed', '') == 'true'

The third line is the one this part will keep coming back to. It is an opt-in condition: a host joins only if it carries an explicit tag saying it should be automated. The alternative — an opt-out, where everything is managed unless tagged otherwise — means a new machine somebody creates with no tags at all is automatically in scope. Lesson 6 is about what happens next.

compose: variables, not groups

compose sets host variables from Jinja expressions. Group membership is not involved.

compose:
patch_window: (provider_tags.env == 'prod') | ternary('sunday-0200', 'anytime')
ansible_host: private_ip_address | default(inventory_hostname)
criticality: provider_tags.get('tier', 'unknown')

The middle line is the most-used compose expression in practice: setting ansible_host from a provider field so Ansible connects to the right address rather than trying to resolve the instance name. It is also the one to check first when a new source produces hosts that are all unreachable.

strict: loud, but not fatal

strict defaults to false, which means an expression that fails — usually because a host is missing the key it references — is skipped and the plugin carries on.

Set strict: true and the same situation ends the parse:

$ ansible-inventory -i inventory/ --graph

[WARNING]: Failed to parse inventory with 'auto' plugin: Failed to parse
'.../10-constructed.yml': Could not set patch_window for host
untagged01.example.com: 'provider_tags' is undefined

<<< caused by >>>

Could not set patch_window for host untagged01.example.com:
'provider_tags' is undefined

Now read that carefully, because it is not what most people assume. strict: true makes the expression failure fatal to the source, and a source that fails to parse is a warning at the top level, not an error. The run continues on whatever the other sources produced, and exits 0.

So strict: true buys you a precise message naming the host and the variable — genuinely valuable, and worth having in development. It does not by itself stop a run. Turning an unparsed source into a hard failure is a separate setting, covered in lesson 8 of this part.

Designing names you would type at 02:00

Group names from a dynamic source are an interface. They are what a person puts after --limit while an incident is running. Four rules, in order of how often violating them causes trouble:

1. Prefix everything. role_, env_, region_, cc_. A prefixed name cannot collide with a hand-written static group, and it says where it came from.
2. Assume the value will be sanitised. Predict eu_west_1 rather than eu-west-1, and confirm with --graph rather than assuming. Every dot and hyphen becomes an underscore.
3. Do not encode two things in one group. role_web_prod forces anyone targeting web servers in staging to invent a second name; role_web plus env_prod lets a pattern intersect them, which is what the : and & operators are for.
4. Name for the reader, not the provider. If the provider tag is Svc and everyone in the room says "service", the group is service_billing. The tag name is an implementation detail of somebody else system.

Rule three is the one that has the most leverage, because it is what makes patterns compose. With role_web and env_prod as separate groups, role_web:&env_prod is the production web tier, role_web:!env_prod is everything else, and neither required anybody to have anticipated the combination.

Knowledge check