Role testing with Molecule in the pipeline · Ansible

Part XXVI covers what a Molecule scenario is, which files make it up on Molecule 26.x, and how to run one while developing a role. This lesson assumes that and asks the pipeline questions: which sequence a CI stage should run, what each stage costs in wall-clock time, and how to choose a test environment that does not manufacture confidence.

The sequence CI should run, and the ones it should not

molecule test runs the full lifecycle — destroy, create, converge, idempotence, verify, destroy — and that is the right thing for a merge gate. It is not the right thing for every stage, and the sub-commands exist because the trade-offs differ.

Command	What it does	Where it belongs
`molecule converge`	Creates the instance if needed and applies the role, leaving it running	A developer’s inner loop. Fast, repeatable, and the instance survives for inspection.
`molecule idempotence`	Re-runs converge against the existing instance and fails if anything reports `changed`	Never alone in CI — it needs a converged instance to exist first
`molecule verify`	Runs the scenario’s assertions against the existing instance	Same; it is a step, not a gate
`molecule test`	The full sequence, destroying at both ends	The merge gate. It is the only one that proves the role works from nothing.

The distinction that matters: converge proves the role works against whatever state the instance is in. test proves it works against a freshly created instance. A role that has been converged eleven times during development is being tested against a machine that already has everything it needs, and that is precisely the machine on which a missing dependency is invisible.

Why idempotence is the highest-value step

Of everything in this part, the idempotence step catches the defect class that nothing else can reach.

ansible-lint can tell you that a shell task has no changed_when. It cannot tell you whether your lineinfile regex matches the line it just wrote, whether your template renders a timestamp, or whether the file mode you set is the one the module then reports as different. Those are run-time facts about the interaction between your task and the machine’s actual state, and the only way to learn them is to run twice and compare.

The mechanism is exactly the “second run is a test” argument from Part XXVI, automated: converge, converge again, fail if the second run reports any changed.

What a failure there actually tells you is worth spelling out, because the first reaction is usually to suppress it:

A command or shell with no creates, removes or changed_when. It reports changed every time by contract. The fix is to describe when it really changed something, not to add changed_when: false and move on — that makes the task lie in the other direction.
A template that renders something non-deterministic. A timestamp, a dictionary iterated in a varying order, a value derived from ansible_date_time. The file genuinely differs on every run.
A regex that does not match what it wrote. lineinfile inserts a line, and next run the regexp fails to recognise it, so it inserts another. Left alone this appends a line per run indefinitely, and the idempotence test is the only thing between you and a config file with four hundred copies of the same directive.
Two tasks fighting. One sets a permission, another resets it. Both report changed forever, and on a real fleet this shows up as a nightly job that never reports zero changes, which is exactly the signal Part XII wants you to be able to trust.

Choosing the environment from what the role touches

On Molecule 26.x there is one driver, and it is called default:

Read-only / Safethe driver list on Molecule 26.6.0— Real output. The historical per-provider drivers are gone; the scenario provisions its own instances.

$ molecule drivers

default

So “container driver versus VM driver” is no longer a configuration setting — it is a property of what your scenario’s create.yml actually provisions. The decision is the same one it always was; only the place you express it has moved.

Make it from what the role touches:

The role manages	Container is honest	Needs a VM
Packages and config files	Yes	No
A user, a directory, file permissions	Yes	No
A systemd unit it enables and starts	Only with systemd as PID 1 in the image, and even then partially	Usually yes
Kernel parameters, modules, `sysctl`	No — the kernel is the host’s	Yes
Filesystems, LVM, mount points	No	Yes
Firewall rules, network interfaces, routing	No — the namespace is not the host’s	Yes
Anything requiring a reboot	No	Yes

This part’s second break/fix: passes in CI, fails on real hosts

A role passed every Molecule run and failed on the first real host.

The container image had no init system as PID 1. The role’s ansible.builtin.systemd_service tasks were skipped or reported success against a service manager that was not managing anything. Enabling the unit, starting it, and the handler that restarted it had never actually been exercised — for months.

Nothing in the CI output said so. A skipped or trivially-succeeding service task looks like a passing service task in a recap.

Two defences:

Assert on the outcome, not the task. The verify step should check that the service is running and listening, not that the task reported ok. Part XII’s outcome-verification lesson is the general form; here it is what distinguishes a real pass from a vacuous one.
Use a real init for roles that manage services. Either an image built to run systemd as PID 1, with the privileges that requires, or a VM. A second scenario using a VM is the honest home for a role that manages a systemd unit and expects a reboot, and it is the one case in this part where a container is not sufficient.

Under the hood

The wall-clock arithmetic is what decides whether the suite survives contact with a real team, and it is worth doing before you have forty roles rather than after.

A single container-based scenario is roughly: create a few seconds, converge as long as the role takes, idempotence the same again, verify a few seconds, destroy a few seconds. The idempotence step doubles the converge time, because it is a second full converge. That is the single largest cost in the sequence and it is the step you least want to remove.

Multiply by platforms and it grows fast: three distributions and two scenarios per role is six runs, and Part XXVI’s platform-matrix lesson covers where to stop. Multiply by forty roles and a full suite is measured in hours.

The structure that keeps it usable:

Only test the roles a change touches. Path-based selection in the pipeline: a change under roles/webserver/ runs the webserver scenarios and nothing else. This is the single biggest win and it is a few lines of pipeline configuration.
Run the full matrix on a schedule, not on every pull request. Nightly against the default branch catches the cross-role and upstream-image regressions that a path-filtered run misses, and it does so without putting hours into the merge path.
Keep VM scenarios out of the merge path unless the change touches the role they cover. They are the slowest thing in the suite by an order of magnitude.

The next lesson assembles all of this into an ordered pipeline with the blocking and non-blocking decisions made explicitly.

What happens if a role genuinely cannot be tested in the pipeline?

A role that provisions storage on a specific SAN, or configures a vendor appliance, or depends on hardware you have one of. These exist, and the usual outcome is a role with no tests at all, which is the worst available answer.

Split it. Almost every untestable role is a testable role with an untestable core:

Test what is testable. The variable validation, the template rendering, the file placement, the service definition. A scenario that runs the role with the hardware-touching tasks tagged out still proves the other eighty per cent.
Make the untestable part a small, obvious task list. If the hardware interaction is three tasks in a separate file, a reviewer can read all of it. If it is scattered through four hundred lines, nobody can.
Assert the preconditions loudly. The role should fail with a clear message when the hardware is absent or in an unexpected state, rather than proceeding. That failure is testable even when the success path is not.
Write down what the tests do not cover, in the role’s README, and treat a change to the untestable part as a change that needs a canary.

The goal is not full coverage. It is knowing precisely where coverage stops, so that the changes crossing that boundary get the human attention the pipeline cannot give them.

The sequence CI should run, and the ones it should not

Why idempotence is the highest-value step

Choosing the environment from what the role touches

Knowledge check