Testing it, then shipping it properly · Ansible

A module that has only ever been tested by running the playbook is a module whose feedback loop is thirty seconds long, involves a managed node, and reports errors as JSON parse failures.

There is a much shorter loop, and there is a test suite that checks several of lesson 3’s contract items automatically. This lesson covers both, then the packaging step that stops the module becoming three divergent copies.

Loop 1: run the module directly

AnsibleModule reads its parameters from a JSON file given as the first argument, so the module can be executed as an ordinary Python script with no ansible-core invocation at all.

Read-only / Safeargs.json - the exact structure AnsibleModule expects— Read-only: file contents. The ANSIBLE_MODULE_ARGS wrapper key is required; the parameters go inside it.

{
"ANSIBLE_MODULE_ARGS": {
  "path": "/tmp/app.conf",
  "key": "max_workers",
  "value": "8",
  "api_token": "REPLACE_ME"
}
}

Configuration changetwo runs, proving idempotency in under a second— Real output from ansible-core 2.21.3. Point path at a scratch file, not a real one - this executes the module for real, with no check mode and no connection layer in the way.

$ python library/app_setting.py args.json; python library/app_setting.py args.json

{"changed": true, "previous_value": null}
{"changed": false, "previous_value": "8"}

That is the fastest possible check of contract item 1, and it is immediate: changed: true then changed: false against the same parameters is exactly what idempotency looks like.

Loop 2: ad-hoc through `ansible-core`

The next loop up adds real argument validation, real check mode and the real execution path, still without packaging anything.

Configuration changerun an unpackaged module ad-hoc— CONFIGURATION without --check; READ-ONLY with it. ANSIBLE_LIBRARY tells ansible-core where to find the module, so nothing needs installing.

export ANSIBLE_LIBRARY=./library

# Check mode: real validation, real check-mode branch, no change.
ansible -m app_setting -a 'path=/tmp/app.conf key=max_workers value=16 api_token=REPLACE_ME' localhost --check

# The full path, including AnsiballZ packing and the remote interpreter.
ansible -m app_setting -a 'path=/tmp/app.conf key=max_workers value=16' localhost -vvv

Read-only / Safecheck mode against a converged target, then a divergent one— Real output from ansible-core 2.21.3. Two invocations: the first requests the value already stored, the second requests a different one. Neither modified the file.

$ ansible -m app_setting -a 'path=/tmp/app.conf key=max_workers value=8' localhost --check

localhost | SUCCESS => {
  "changed": false,
  "previous_value": "8"
}

localhost | CHANGED => {
  "changed": true,
  "previous_value": "8"
}

Contract item 3 in two commands: the module reports what it would do and does nothing. That second result — CHANGED under --check with the file untouched — is what real check-mode support looks like.

Loop 3: `ansible-test sanity`

The sanity suite is the only automated check of several contract items, and it ships with ansible-core. It has one structural requirement that stops people on first use.

Read-only / Saferunning ansible-test outside a collection tree— Real output from ansible-core 2.21.3. The collection must live at a path ending .../ansible_collections/<namespace>/<name>/ - the parent directory name is part of the contract, not a convention.

$ ansible-test sanity --test validate-modules plugins/modules/app_setting.py

FATAL: The current working directory must be within the source tree being tested.

Testing an Ansible collection: {...}/ansible_collections/{namespace}/{collection}/
Example #1: community.general -> ~/code/ansible_collections/community/general/

No "ansible_collections" parent directory was found.

With the tree arranged correctly, the suite reports real defects:

Read-only / Safevalidate-modules on the module from lesson 2— Real output from ansible-core 2.21.3. Three findings on a module that runs perfectly. ansible-test creates its own isolated virtualenv for the test requirements on first run.

$ ansible-test sanity --test validate-modules plugins/modules/app_setting.py

ERROR: Found 3 validate-modules issue(s) which need to be resolved:
ERROR: plugins/modules/app_setting.py:0:0: missing-examples: No EXAMPLES provided
ERROR: plugins/modules/app_setting.py:0:0: missing-gplv3-license: GPLv3 license header not found in the first 20 lines of the module
ERROR: plugins/modules/app_setting.py:0:0: no-log-needed: Argument 'key' in argument_spec could be a secret, though doesn't have `no_log` set
FATAL: The 1 sanity test(s) listed below (out of 1) failed. See error output above for details.
validate-modules

Read those three, because each teaches something different.

missing-examples is a documentation gap that costs nothing at run time and everything to the colleague reading ansible-doc at 03:00.

missing-gplv3-license is a licensing requirement for collection modules. It is mechanical, and it matters if the collection ever leaves your organisation.

no-log-needed on key is the interesting one, and it is a false positive. The check flags any argument whose name pattern suggests a credential — key does, even though here it holds a setting name. The resolution is to declare the intent explicitly:

Read-only / Safeanswering no-log-needed honestly— Read-only: source fragment. no_log=False is not the default-by-omission; it is an explicit statement that this argument was considered and is not a secret.

argument_spec=dict(
  path=dict(type='path', required=True),
  key=dict(type='str', required=True, no_log=False),
  value=dict(type='str', required=True),
  api_token=dict(type='str', required=False, no_log=True),
),

With EXAMPLES added, the licence header present and no_log=False declared, the same command passes:

Read-only / Safethe same module after the three fixes— Real output from ansible-core 2.21.3, exit code 0. No findings.

$ ansible-test sanity --test validate-modules plugins/modules/app_setting.py; echo exit=$?

Running sanity test "validate-modules"
exit=0

The full ansible-test sanity run adds pep8, pylint, yamllint, import checks and several more. Start with validate-modules because it is the one that maps onto lesson 3’s contract, then add the rest.

Shipping it: a collection, not a copy

library/ is right for development and wrong for distribution, and the failure is entirely predictable.

The module is useful, so a second repository needs it, so somebody copies library/app_setting.py. Then a third. A bug is found and fixed in one. Eighteen months later there are three files with the same name, different behaviour, and no way to tell which one a given play ran.

A collection gives the module a namespace, a version and a single source.

Configuration changecreate the collection and place the module in it— CONFIGURATION: creates a directory tree on the workstation. The ansible_collections/<namespace>/<name>/ path is required by ansible-test, so build the tree that way from the start.

mkdir -p ~/code/ansible_collections/example
cd ~/code/ansible_collections/example

ansible-galaxy collection init example.fleet --init-path .

mkdir -p fleet/plugins/modules
cp /srv/ansible/library/app_setting.py fleet/plugins/modules/

Read-only / Safegalaxy.yml - the metadata that makes it a versioned artefact— Read-only: file contents. version is the field that turns a directory of code into something requirements.yml can pin, which is what Part L lesson 3 depends on.

namespace: example
name: fleet
version: 1.0.0
readme: README.md
authors:
- Example Team <ops@example.com>
description: Internal modules for the example.com fleet
license:
- GPL-3.0-or-later
repository: https://git.example.com/ansible/collection-fleet
tags:
- internal

Configuration changebuild the distributable artefact— Real output from ansible-core 2.21.3, absolute path shortened. CONFIGURATION: writes a tarball to the output path. This file is what you publish to an internal Galaxy server, or store alongside the escrow material from Part L lesson 4.

$ ansible-galaxy collection build --output-path ./dist

Created collection for example.fleet at /home/ansible/code/dist/example-fleet-1.0.0.tar.gz

The module is now example.fleet.app_setting in a play, and pinnable in requirements.yml:

Read-only / Safeconsuming the collection from a project— Read-only: file contents. The pin is the point - a rebuilt controller gets this exact version rather than whatever is newest.

---
collections:
- name: example.fleet
  version: 1.0.0
  source: https://galaxy.example.com/api/galaxy/content/internal/

Under the hood

Why does ansible-test insist on the ansible_collections/ parent directory rather than accepting a path argument?

Because it constructs a real collection loading environment to run the tests in, and the loader finds collections by walking a path whose structure is the namespace. ansible_collections/example/fleet/ resolves to example.fleet; there is no metadata file consulted to establish that mapping — the directory names are the identity.

Two things follow. First, the same rule applies at runtime: a collection installed anywhere is found by that path shape, which is why collections_path in Part L lesson 3 points at a directory containing ansible_collections/, not at the collection itself. Second, galaxy.yml does not determine where the collection can live; it records what to build. A mismatch between the directory names and the namespace/name in galaxy.yml produces a collection that builds under one identity and loads under another, which is a genuinely confusing afternoon.

The sanity tests then run inside an isolated virtualenv that ansible-test creates and populates itself, from a pinned requirements file per test. That is why the first run downloads packages and later runs do not, and it is why the tests do not disturb the environment you develop in. It is also why the suite can assert things about your code’s imports: it controls exactly what is importable.

What happens if you cannot run an internal Galaxy server?

Nothing about collections requires one. ansible-galaxy installs from several sources, and three of them need no server at all.

From a Git repository. requirements.yml accepts a git+https:// source with a version: that is a tag or a commit. The pin is a Git ref, which is as reproducible as a version number and arguably more so.

collections:
  - name: git+https://git.example.com/ansible/collection-fleet.git
    type: git
    version: v1.0.0

From a built tarball. ansible-galaxy collection install ./dist/example-fleet-1.0.0.tar.gz installs the artefact directly. Store the tarballs in an artefact repository you already run — most organisations have one for other languages — or alongside the DR material from Part L.

From a directory. For a monorepo, a collection can sit in the tree and be found through collections_path, with no install step at all. This trades the version pin for immediacy, which is the right trade during development and the wrong one across repositories.

The thing to avoid is the fourth option, which is to conclude that because there is no server, library/ will do. The value of the collection is the namespace, the version and the single source of truth — and all three are available from a Git tag.

Knowledge check

Why this matters

Two habits separate a module that survives from one that becomes a liability, and neither is about the code.

The first is a short feedback loop. A module tested only by running the playbook is tested rarely, because each attempt costs thirty seconds and a managed node. The JSON args file turns that into under a second, which changes how often you check idempotency from “once, at the end” to “after every edit”. Contract item 1 becomes something you observe rather than something you intend.

The second is packaging before the second consumer appears. The moment somebody copies the file is the moment you acquire a maintenance problem that grows silently — and it is entirely preventable by a galaxy.yml, a version number and a Git tag, none of which require infrastructure you do not already have.

Both are cheap at the start and expensive to retrofit. A module that has been in library/ across three repositories for two years cannot be consolidated without reconciling three behaviours, and nobody knows which one production depends on.

Loop 1: run the module directly

Loop 2: ad-hoc through ansible-core

Loop 3: ansible-test sanity

Shipping it: a collection, not a copy

Knowledge check

Loop 2: ad-hoc through `ansible-core`

Loop 3: `ansible-test sanity`