Not everything runs on the target

The module contract says a module runs on the target. That is true, and several of the things you write in a task file are not modules.

ansible.builtin.template looks exactly like a module in a playbook. It has options, it reports changed, it appears in ansible-doc -l. But the Jinja rendering happens on your controller, using your controller’s filesystem, before anything is sent anywhere. The thing that does that is an action plugin, and it is what the task actually invokes.

This one distinction explains a whole family of otherwise unrelated confusions.

The two-layer picture

A task with a module name goes through two layers:

1. The action plugin, which runs on the controller. It may do preparatory work — render a template, read a local file, compute a checksum, decide whether to call a module at all.
2. The module, which is packaged, copied to the target, executed there, and returns JSON.

Most tasks use the generic normal action plugin, which does nothing interesting: it packages the module and runs it. Those tasks are the simple case, and the mental model “the module runs on the target” is completely correct for them.

Twenty-nine modules in ansible-core 2.21 declare an action attribute, meaning they have a bespoke action plugin doing controller-side work first. These are the ones that break the simple model:

add_host   assemble    assert      async_status  command
copy       debug       dnf         fail          fetch
gather_facts  group_by  include_vars  normal     package
pause      raw         reboot      script        service
set_fact   set_stats   shell       template      unarchive
uri        validate_argument_spec  wait_for_connection

$ ansible-doc ansible.builtin.command | grep -A1 'note:'

  * note: This module has a corresponding action plugin.

The action attribute in the attributes table says the same thing more precisely, with a support level:

action: Indicates this has a corresponding action plugin so some parts of the options can be executed on the controller.

Where the work happens, module by module

The last row is the sharpest one. debug, assert, fail and set_fact never touch the target at all.

The src that must exist on the controller

The most common consequence in day-to-day work is that template, copy, script and assemble resolve src on the controller.

Watch what happens when the file is missing:

$ ansible-playbook ctrl.yml --check

TASK [Template a source that does not exist on the controller] *****************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Could not find or access 'templates/nowhere.conf.j2'\nSearched in:\n\t/opt/estate/templates/nowhere.conf.j2\n\t/opt/estate/templates/nowhere.conf.j2 on the Ansible Controller."}

“on the Ansible Controller” is in the error text verbatim. Ansible is being explicit because this is the misdiagnosis it expects: the reader sees a path, SSHes to the target, finds the file present, and concludes Ansible is broken.

The mirror image is fetch, where dest is on the controller. A fetch that fails with a permission error is usually a permission error on your workstation, not on the fleet.

Lookups read the controller too

lookup() is a plugin family that executes on the controller during templating. So:

- name: This reads a file on the CONTROLLER
  ansible.builtin.debug:
    msg: "{{ lookup('ansible.builtin.file', '/etc/hostname') }}"

- name: This reads a file on the TARGET
  ansible.builtin.slurp:
    src: /etc/hostname

Two tasks, the same path, two different machines. The first prints your controller’s hostname for every host in the play — identically, because it is the same file read once per templating pass.

lookup('pipe', ...) and lookup('env', ...) are the same story and more dangerous: pipe runs a shell command on the controller. A task that reads lookup('pipe', 'hostname') inside a fleet-wide play executes hostname on your workstation, once per host, and gives you your own hostname 300 times.

delegate_to: localhost is a third thing

Worth naming, because it is easy to conflate with the above.

delegate_to: localhost does not make a task controller-side in the action-plugin sense. It changes which host the task connects to, so the module still runs — packaged, copied, executed — but on the controller acting as a target.

The distinguishing detail: variables still come from the original host in the play loop. A task delegated to localhost inside a play over 300 hosts runs 300 times, on your controller, each with a different host’s variables. That is usually what you want for something like an API call per host, and it is a common way to accidentally run something 300 times that you meant to run once. run_once: true is the pairing that fixes it.

Knowledge check