Where container fidelity stops

A role that passes in a container is not thereby production-safe on a VM.

If you take one sentence from this part, take that one. Everything else in this lesson is the detail behind it: which roles the statement bites on, why the container run still looked green, and what to do about the gap other than pretend it is not there.

Containers are the right default for testing automation. They start in seconds, they cost nothing, they are genuinely disposable, and they let a role be exercised end to end on three distributions in the time a single VM takes to boot. None of that is in question. The question is what the resulting green tick is a statement about.

What a container models faithfully

More than people expect, which is why the boundary is easy to miss.

• Package installation. apt, dnf and zypper work normally. A role that installs the wrong package name for Debian fails in a Debian container exactly as it would on a Debian VM.
• Files, templates and permissions. Rendering, ownership, mode, SELinux file contexts where the container carries the policy.
• Users and groups. /etc/passwd, /etc/group, home directories, shells.
• Role logic. Conditionals, loops, variable precedence, handler wiring, block/rescue, tags, dependency ordering. This is a large fraction of what a role is, and it is fully exercised.
• Idempotency of everything above. The second-run test from lesson 2 is meaningful in a container, for the tasks a container can run.
• Distribution differences in paths and package names. /etc/nginx/ versus /etc/nginx/conf.d/, httpd versus nginx, apache2 versus httpd. This is the single most valuable thing a container matrix gives you, and it is cheap.

That is a real test suite. A role that passes all of it has been shown to be internally coherent, portable across the distributions you tested, and free of the whole class of defect that comes from writing a role against one distribution’s filesystem layout.

Where it stops

The boundary is not arbitrary. A container shares the host kernel and usually does not run an init system, so anything that depends on either of those is either absent, simulated, or quietly skipped.

Some of these can be partially papered over: a systemd-enabled image run with the right capabilities and a cgroup mount can boot systemd as PID 1 inside a container, and that recovers unit management. It does not recover reboot, kernel, storage or the host network, and it costs privileged or a carefully chosen capability set — which is a real security decision, not a test detail.

The failure mode that makes it look green

The dangerous case is not the role that fails in a container. It is the role that passes because the interesting task never ran.

Consider a task written the way careful people write it:

- name: Ensure nginx is running and enabled
ansible.builtin.systemd_service:
  name: nginx
  state: started
  enabled: true
when: ansible_service_mgr == 'systemd'

The when: guard is there for a good reason — the role is supposed to work on hosts that do not use systemd — and it is also the thing that makes the container run meaningless. In a container whose PID 1 is sleep, ansible_service_mgr is not systemd, the condition evaluates false, and Ansible reports:

$ molecule converge

TASK [webserver : Ensure nginx is running and enabled] **************************
skipping: [instance]

PLAY RECAP *********************************************************************
instance                   : ok=11   changed=4    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0

failed=0. The scenario passes. The idempotence step passes, because a skipped task is skipped on both runs. And the one thing the role exists to do has never been executed anywhere.

Then the role runs on a VM, where the condition is true, and the task runs for the first time in its life against a real service manager. If the unit file the role installed has a typo in ExecStart, that is when you find out — in production, or in a staging window that everyone thought was a formality.

This is the break/fix scenario for this part, and the diagnosis people reach for first is wrong. The instinct is “the container was broken”. The container did what it was asked. What was broken was the assumption that a green scenario meant the role had been tested.

Deciding where a role belongs

The useful question is not “container or VM?” in the abstract. It is: what does this role depend on that a container does not have?

The costs are real in both directions. A VM takes minutes to provision where a container takes seconds; a matrix of three distributions on VMs is a meaningfully slower pipeline and a meaningfully larger infrastructure bill. That is why the answer is a split rather than a preference: run the fast container matrix on every commit, and run the VM tests for the roles in the bottom half of that table on a schedule and before a release.

Writing the gap down

The final habit, and the one most often skipped: a role’s testing status should say what was not tested.

## Test fidelity

Verified in containers (Ubuntu 24.04, Debian 12, Rocky 9):
package installation, config rendering, file permissions,
role logic and idempotency.

NOT verified in containers, and therefore not verified on every commit:
- unit ordering and restart policy (no systemd as PID 1)
- behaviour across a reboot
- the sysctl settings in tasks/tuning.yml (namespaced in a container)

Covered by the VM scenario, which runs nightly and before a release:
unit ordering, restart policy, reboot survival.

Not covered anywhere before production:
behaviour under load, and the log volume the access-log settings
produce on a real traffic profile.

That block takes five minutes to write and answers the only question that matters during an incident review: did we know?

Knowledge check