Machine-readable run artefacts · Ansible

The controller log from the previous lesson has one structural flaw that no amount of retention fixes: it is a rendering. It contains the text a callback produced for a human, which means answering a question about it requires parsing prose that changes shape when somebody sets callback_result_format.

An artefact is different. It is the same events, emitted as a data structure, with the per-task and per-host detail intact. You can ask it questions with a program.

The distinction matters most at exactly the moment you need it: months later, when nobody remembers the run and the question is precise.

What is available, and where it lives

ansible-core ships two callbacks that write files, and both come with caveats worth knowing before you build on them:

Read-only / Safethe file-writing callbacks in core— Verified on ansible-core 2.21.3 in a virtualenv with no collections installed.

$ ansible-doc -t callback -l

ansible.builtin.default default Ansible screen output
ansible.builtin.junit   write playbook output to a JUnit file
ansible.builtin.minimal minimal Ansible screen output
ansible.builtin.oneline oneline Ansible screen output
ansible.builtin.tree    Save host events to files

ansible.builtin.junit writes a JUnit XML file — one test case per task per host, with the module result as system-out. It is genuinely useful and its main consumer is a CI system that already understands JUnit. Verified on 2.21.3, the top of a real file:

Read-only / Safea real JUnit artefact from the four-host recap play— Verified on ansible-core 2.21.3. Hosts use ansible_connection=local and the play contains only debug and fail tasks; the file was written into a scratch directory.

$ head -8 junit/recap-1786491185.8839395.xml

<?xml version="1.0" ?>
<testsuites disabled="0" errors="2" failures="0" tests="25" time="0.2650287151336669921875">
<testsuite disabled="0" errors="2" failures="0" name="recap" skipped="12" tests="25" time="0.265028715133666992">
	<testcase classname="/path/to/recap.yml:5" name="[web1] Every recap field in one play: Plain ok msg=ok" time="0.0104944705963134765625">
		<system-out>{
  "changed": false,
  "msg": "ok"
}</system-out>
	</testcase>

Note the shape of that: 25 test cases for a four-host play with seven tasks, because the unit is task-times-host. Note also that the module result is embedded verbatim — including anything a task returned that no_log did not suppress, which makes this file exactly as sensitive as the log from the previous lesson.

ansible.builtin.tree writes one file per host, and it is both deprecated for removal in 2.23 and structurally limited. Verified on 2.21.3, the file it produced for a host that ran seven tasks:

Read-only / Safetree keeps the last result per host, not the run— Verified on 2.21.3. The play ran seven tasks against this host; the file contains one result.

$ cat tree/web1

{
  "changed": false,
  "msg": "conditional"
}

One result. It is a per-host snapshot, not a run record, and it is on its way out. Do not build on it.

The JSON callbacks are in ansible.posix. ansible.posix.json emits a single JSON document for the run; ansible.posix.jsonl emits one JSON object per line, which is what a log pipeline wants because it can be appended and streamed. Both names were confirmed against the collection documentation.

Their output is not reproduced here. The validation controller for this course runs a bare ansible-core with no collections installed, and this course does not print output it did not produce. Install ansible.posix, pin it in requirements.yml, and enable it with callbacks_enabled.

[defaults]
callbacks_enabled = ansible.posix.jsonl

The artefact does not know what the run was

Here is the failure that makes most run artefacts useless, and it is not a tooling problem.

Every callback consumes the event stream: tasks starting, results arriving, the run ending. None of them is told the things a later investigation actually needs:

Which repository commit the playbook came from.
Which inventory source was used, out of the several the project has.
Which --limit was applied — the single most important one.
Which vault ids were unlocked.
Who ran it, and from where.
Whether it was --check.

None of that is in the artefact by default, because none of it is an event. And a run record that says “site.yml ran, 180 hosts changed” without saying which limit produced those 180 hosts cannot answer “which hosts did this touch?” — because, as lesson 1 established, hosts that were never reached do not appear in the recap at all.

Stamping the run with its own identity

ansible.builtin.set_stats is the mechanism, and it is in core. It attaches arbitrary data to the run’s statistics, which every callback then sees.

- name: Record what this run is
  hosts: localhost
  connection: local
  gather_facts: false
  tasks:
    - name: Stamp the run with its identity
      ansible.builtin.set_stats:
        data:
          repo_commit: "{{ lookup('ansible.builtin.env', 'GIT_COMMIT') | default('unknown', true) }}"
          inventory_source: "{{ ansible_inventory_sources | join(',') }}"
          limit_applied: "{{ ansible_limit | default('none', true) }}"
          check_mode: "{{ ansible_check_mode }}"
          operator: "{{ lookup('ansible.builtin.env', 'SUDO_USER') | default(lookup('ansible.builtin.env', 'USER'), true) }}"
        aggregate: false

Put that play first in site.yml, and every run carries its own provenance. With custom stats displayed, it appears in the output — verified on 2.21.3:

Read-only / Safethe run stamped with its own identity— Verified on ansible-core 2.21.3. The play targets localhost with connection: local and its only task is set_stats, which changes nothing on any system. Values shown are obvious placeholders.

$ ANSIBLE_SHOW_CUSTOM_STATS=1 ansible-playbook -i hosts.ini meta.yml

PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0

CUSTOM STATS: ******************************************************************
RUN: { "inventory_source": "inventory/production", "limit_applied": "webservers:!web4", "operator": "REDACTED", "repo_commit": "REPLACE_ME_COMMIT_SHA"}

show_custom_stats is an option of the default callback (ANSIBLE_SHOW_CUSTOM_STATS, or show_custom_stats in [defaults]), and the same data reaches any structured callback you have enabled.

ansible_limit and the magic variables it depends on

ansible_limit, ansible_inventory_sources and ansible_check_mode are magic variables — they exist because the engine sets them, not because a fact was gathered. All three were verified on 2.21.3, and one of them has a trap:

Variable	Observed value
`ansible_limit`	the limit string exactly as given, e.g. `web:!web4`
`ansible_limit` with no `--limit`	undefined, not an empty string
`ansible_inventory_sources`	a list of absolute paths
`ansible_check_mode`	`True` under `--check`, `False` otherwise

The undefined case is why default('none', true) appears in the stamping play. Without it, a run with no limit fails on an undefined variable — in the first play of site.yml, before anything useful has happened.

The default('unknown', true) on the environment lookups is deliberate. Without the second argument, default only substitutes when the value is undefined; an environment variable that exists and is empty passes through as an empty string. In a provenance record, "" and “unknown” mean different things to the person reading it in six months.

Sidecar metadata: the belt-and-braces version

The stamping play covers the run’s own view of itself. A wrapper — a Makefile target, a CI step, a thin shell script that everyone uses instead of typing ansible-playbook — can record things the run cannot see:

Read-only / Safea sidecar written before the run starts— Collects provenance and writes it to a file. Runs no play and touches no managed host.

RUN_ID="$(date -u +%Y%m%dT%H%M%SZ)-$$"
mkdir -p artefacts
jq -n --arg id "$RUN_ID" --arg commit "$(git rev-parse HEAD)" --arg branch "$(git rev-parse --abbrev-ref HEAD)" --arg dirty "$(git status --porcelain | wc -l)" --arg host "$(hostname -f)" --arg user "${SUDO_USER:-$USER}" '{run_id: $id, commit: $commit, branch: $branch, uncommitted_files: $dirty, controller: $host, operator: $user}' > "artefacts/${RUN_ID}.meta.json"

uncommitted_files is the field people leave out and then need. A run made from a working tree with local modifications is not reproducible from the commit, and the honest artefact says so.

The artefact is as sensitive as the log

A structured artefact embeds module results verbatim. That includes the contents of files that a --diff rendered, the output of command tasks, and any registered variable a task returned.

Everything from the previous lesson applies unchanged: audit for tasks that return secret material and fix them with no_log before you start writing artefacts, create the artefact directory with a restrictive mode, and decide retention deliberately.

The addition here is that artefacts are usually shipped somewhere — a CI artefact store, an object bucket, a log pipeline. That moves the data outside the controller’s access controls, and “who can read the CI artefacts of the infrastructure repository” is frequently a much larger set of people than “who can log in to the controller”.

Retention is a decision, not a default

Three questions, answered once, written down:

How long? Driven by the longest question you expect to answer. “Which automation touched this host before the incident” is usually answered within days; “who changed this control before the audit period closed” is quarters. Pick the longer one and say why.
How much? An artefact per run, per host, per task grows with the fleet and the schedule. A nightly run over 400 hosts with 200 tasks is 80,000 records a night. Estimate before enabling, not after the bucket bill.
What is deleted, and can it be? If your retention policy says ninety days and the artefacts contain something you are required to keep for seven years, you have a conflict to resolve now rather than during the audit.

The failure mode of getting this wrong is not usually cost. It is that the artefacts stop being written when the disk fills, quietly, and nobody notices until the first time somebody needs one.

Under the hood

Why does set_stats reach every callback, when a registered variable does not?

Because statistics are a run-level object rather than a host-level result. The engine maintains an AggregateStats structure for the whole run — the same object that accumulates the per-host ok, changed and failed counters — and set_stats writes into its custom section. At the end of the run, v2_playbook_on_stats hands that whole object to every enabled callback.

A registered variable, by contrast, lives in the variable manager for one host and never becomes an event at all. A callback never sees it, which is why “just register the commit and print it” does not produce a record.

The aggregate parameter controls what happens when set_stats runs more than once with the same key. With aggregate: true, which is the default, numeric values accumulate across invocations — useful for counting something across a play, and wrong for provenance, where a second invocation should replace rather than add. Hence aggregate: false in the stamping play above.

One consequence worth planning for: because custom stats are run-level, they are not per host. You cannot use them to record something different about each host; that belongs in the per-host results the structured callback is already emitting. Custom stats answer “what was this run”, and the task results answer “what happened to this host”. A useful artefact needs both, joined by the run id.

What happens if the artefact records the playbook but not the limit?

You cannot answer the only question that matters after an incident, and the artefact will look complete while failing to.

The scenario is exact. A change goes out on Tuesday. On Thursday a host misbehaves and someone asks: did Tuesday’s run touch this host?

You have the artefact. It says site.yml ran at 14:02, 180 hosts reported results, 47 of them changed. The host in question is not in the list.

That is not an answer. Two entirely different situations produce it:

The run was limited to a group this host is not in. It was never a target. Tuesday is irrelevant to Thursday’s problem.
The run targeted everything, the play failed early, and this host was in a batch that was never reached — so it is absent from the recap for the same reason, while being very much a target that did not get its change. That is a host now running the old configuration in a fleet that has moved on, which is a completely different incident.

Without the --limit and the intended target set, you cannot tell those apart. With them, it is a set subtraction: intended minus reported equals untouched.

The fix is one field, recorded once, and it has to be recorded at run time — --limit is a command-line argument that exists nowhere afterwards. This is why the stamping play is first in site.yml rather than a documentation note, and why the sidecar records the command line as invoked.

Knowledge check

Why this matters

There is a specific moment this lesson is written for. Someone from outside your team asks a precise question about a change made months ago, and the answer has to be evidence rather than recollection.

At that moment the difference between a log and an artefact is the difference between reading and searching, and the difference between a complete artefact and an incomplete one is the difference between an answer and a shrug. The fields that turn out to matter are never the exciting ones — they are the commit, the inventory, the limit and whether the working tree was clean.

None of that is captured for you, and no callback can capture it, because none of it is an event. It is context about the invocation, and the only place to record it is inside the invocation. That is why the stamping play sits first in site.yml and the wrapper writes a sidecar: two cheap habits that convert a pile of run output into something that can be interrogated.

What no artefact can supply on its own is the identity of the human. Every record so far says ansible — the service account. The next lesson is about what it takes to put a person’s name back on a change.