What a clean --check does not prove · Ansible

This lesson exists because of a sentence that gets said in change reviews, in this form or a close variant:

We ran it with --check and it came back clean.

It is offered as evidence that the change is safe, it is accepted as evidence that the change is safe, and it is not evidence that the change is safe. The documentation is unusually direct about why:

Check mode is just a simulation. It will not generate output for tasks that use conditionals based on registered variables (results of prior tasks).

Four classes of defect pass a clean check run. Each one is demonstrated below on ansible-core 2.21.3.

1. Tasks that were never evaluated

Modules without check-mode support are, in the documentation’s words, “modules that do not support check mode report nothing and do nothing”. They come back as skipping, which is the same word a false when: produces.

The consequence for reading a recap: skipped is a count of two completely different things, and the modules most likely to be in it — command, shell, script, raw — are the ones doing the work you most wanted previewed. The module-model part covers the support levels and how to look them up before a change window; the point here is narrower and worse.

2. A play whose later tasks depend on its earlier effects

This is the class the documentation sentence is about, and it is the one that produces confidently wrong output rather than missing output.

- name: What a dry run does to a play that depends on its own effects
  hosts: appservers
  tasks:
    - name: Create the marker the rest of the play depends on
      ansible.builtin.command: "touch {{ marker }}"

    - name: Look for the marker
      ansible.builtin.stat:
        path: "{{ marker }}"
      register: marker_stat

    - name: Decide what to do next based on the marker
      ansible.builtin.debug:
        msg: "marker present: {{ marker_stat.stat.exists }}"

    - name: Only run when the marker is present
      ansible.builtin.debug:
        msg: "This is the task that does the real work."
      when: marker_stat.stat.exists

Read-only / Safethe dry run that lied— Verified on ansible-core 2.21.3. Read the recap: no failures, no changes, exit code 0. Then read what happened.

$ ansible-playbook -i inv.ini marker.yml --check; echo "exit=$?"

TASK [Create the marker the rest of the play depends on] ***********************
skipping: [localhost]

TASK [Look for the marker] *****************************************************
ok: [localhost]

TASK [Decide what to do next based on the marker] ******************************
ok: [localhost] => {
  "msg": "marker present: False"
}

TASK [Only run when the marker is present] *************************************
skipping: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

exit=0

failed=0, changed=0, exit 0. On a dashboard that is a clean run.

What actually happened: task one was skipped because command cannot predict an arbitrary program. Task two then measured a world in which task one had not run, and correctly reported that the marker is absent. Task three printed False. Task four — the task that does the real work — was skipped on the strength of it.

The check run reported on a hypothetical world: the one where the first task does not happen. That world will never exist. In the real run task one creates the marker, marker_stat.stat.exists is true, and the task that check mode never evaluated is the one that runs.

Check mode is least informative exactly where a play is most sequential

The severity of this scales with how much a play depends on its own progress, which is to say with how much it resembles a deployment.

A play that installs a package, writes a config into a directory the package creates, and restarts a service is three sequential dependencies deep. Its check run reports on: the install (predicted), the config write (failed, because the directory does not exist yet), and the restart (not reached).

The routine reading of that output is “check mode does not work for this play”, followed by dropping check mode. The accurate reading is “this play cannot be usefully dry-run in its current shape”, which is a statement about the play and is actionable. The next lesson is about which of those two responses you take.

3. Preconditions the module does not evaluate in check mode

The previous class at least leaves a visible trail of skips. This one produces a check run that is not merely uninformative but actively encouraging.

Read-only / Safethree changes predicted, with a diff— Verified on ansible-core 2.21.3 against a scratch path, shown here with a documentation path substituted. The destination directory does not exist; check mode was not asked to notice.

$ ansible-playbook -i inv.ini render.yml --check --diff; echo "exit=$?"

TASK [Render the application config] *******************************************
--- before
+++ after: /etc/app/app.conf
@@ -0,0 +1,4 @@
+[database]
+host = db01.example.com
+user = appuser
+password = REDACTED

changed: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=3    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

exit=0

A reviewer would approve that. Three predicted changes, a readable diff of exactly what will be written, nothing skipped, exit 0.

Here is the same play, run for real, with nothing else changed:

Configuration changethe real run, immediately— Verified on ansible-core 2.21.3 against a path in a scratch directory. The first task fails; no file is written.

$ ansible-playbook -i inv.ini render.yml

TASK [Render the application config] *******************************************
fatal: [localhost]: FAILED! => {"changed": false, "checksum": "12abdd31cf62e1aa0544a8ee68e9bddeef51ff5f", "msg": "Destination directory /etc/app does not exist"}

PLAY RECAP *********************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

copy in check mode compares the content it would write against the content that is there, decides they differ, and reports changed. It does not verify that the destination directory exists, because verifying the environment is not what check mode is for — check mode answers would this produce a change, not would this succeed.

Those are different questions and the output does not distinguish them. A predicted change is a statement about desired state, not a prediction of success.

4. Everything second-order

The class that no amount of module support could ever address.

Check mode tells you a config file would change. It cannot tell you the service will start with the new content. It cannot tell you the new value is correct, that the port is free, that the certificate matches the hostname, that the downstream service can parse what you are about to give it, or that four hundred hosts restarting within the same minute will not overwhelm the thing behind them.

--diff narrows this usefully — it shows the content rather than the fact of a change — but “these three lines will change” is a statement about the file, and every consequence of the change lives outside the file.

What a clean check run does establish

This lesson is not an argument for skipping check mode. A green --check run proves a specific and worthwhile list:

The controller can reach and authenticate to every targeted host, and privilege escalation works. That alone catches a large fraction of change-window failures.
The inventory resolves to the hosts you expect. The recap is a host list.
Guards fire. assert has full check-mode support, so a guarded play is guarded in its dry run.
Variables and templates resolve for every task that was evaluated. An undefined variable in a rendered template shows up here.
The tasks with real check-mode support report a change count, and with --diff, the content of it.
Handlers are notified and run, exercising the notification chain.

That is a good rung on the ladder. It is simply not the rung that answers “is this safe”, and the reason to be precise about it is that the sentence “check mode passed” is doing that work in change reviews today.

Why the skipped result is worse than a missing one

The mechanism behind class 2 is worth seeing exactly, because the intuition is wrong in a way that matters.

When a command task is skipped in check mode, it returns a result dictionary. Registered, it looks like this:

{
  "changed": false,
  "cmd": ["/bin/echo", "would-have-run"],
  "failed": false,
  "msg": "Command would have run if not in check mode",
  "rc": 0,
  "skipped": true,
  "stdout": "",
  "stdout_lines": []
}

A downstream conditional therefore does not raise an undefined-variable error. It evaluates against those values:

Condition	Real run	Check run	What that means
`when: probe.rc == 0`	true	true	for a command that never ran
`when: probe.stdout == '4.2.1'`	true	false	wrong branch, silently
`when: probe.stdout is defined`	true	true	and meaningless
`when: not probe.skipped`	true	false	the honest test

Only the last one distinguishes them, and almost nobody writes it.

This is the mechanical explanation for the documentation’s warning about conditionals based on registered variables. It does not say the play errors — it says check mode will not generate output for those tasks, which is a careful way of saying the branch you see taken is not the branch that will be taken.

If you must reason about a registered result in a play that will be dry run, when: not probe.skipped | default(false) at least makes the difference visible, and check_mode: false on a genuinely read-only probe removes the problem instead of documenting it.

What happens if the check run fails partway through?

That is usually the honest outcome, and treating it as a defect in check mode is how teams talk themselves out of using it.

A bootstrapping play, a deployment, anything with sequential dependencies: the check run is expected to fail at the first task that needed a previous task’s effect. The correct reading is “check mode validated tasks one through eight, and cannot say anything about nine onwards”, which is genuine information about a real boundary in the play.

Three responses, in order of preference:

Accept it and record the boundary. Note in the change record which task the check run reached. That number is the extent of what was validated, and stating it is more useful than a green tick.
Restructure so the check run gets further. check_mode: false on read-only probes, and moving prerequisites into a separate play that has already run. The next lesson is entirely about this.
Conclude that this play cannot be usefully dry-run and use a canary. A legitimate answer for some plays, and much better than pretending a truncated check run validated the whole thing.

The response to avoid is the fourth one: adding ignore_errors: true or check_mode: false until the check run goes green. That converts an honest partial validation into a dishonest complete one, and the whole point of this lesson is what people then do with the word “clean”.

Knowledge check

Why this matters

The failure mode this lesson guards against is not technical. It is a sentence in a meeting.

“Check mode passed” is short, sounds rigorous, and is almost never challenged, because challenging it requires knowing which modules in the play support check mode and which tasks depend on earlier effects — which the person saying it usually does not know either. The claim travels further than the evidence behind it, and it travels into the record that authorised the change.

The correction is not to distrust check mode. It is to say the more precise thing, which is barely longer: “the dry run reached task fourteen of twenty, predicted changes to three files on twelve hosts, and skipped the four command tasks.” That sentence tells a reviewer what was and was not examined, and it is a sentence anybody can say once they have read the output rather than the recap.