Lookups that touch secrets · Ansible

Lesson 2 established that lookups run on the controller. That is the machine holding SSH keys to the whole fleet and the vault password that decrypts your secrets — so anything that reads secrets, or executes code, does it there, with your privileges.

This lesson covers the five ansible.builtin lookups that handle credential material, and it spends most of its length on the two that behave in ways people do not expect.

The five

Lookup	Reads	Watch for
`env`	A controller environment variable	Absent variables yield empty, not an error
`file`	A controller file	Vault-encrypted files come back as ciphertext
`unvault`	A vault-encrypted controller file, decrypted	Needs the vault password available
`password`	Generates a password — and writes it to a file	The side effect
`pipe`	The output of a shell command on the controller	It is a shell

`env`

Read-only / Safereading CI context

- ansible.builtin.debug:
  msg: "deploying {{ lookup('ansible.builtin.env', 'CI_COMMIT_SHA') }}"

Right for anything the pipeline puts in the environment. Two properties to hold on to: a variable that is not set yields an empty string rather than failing, and the environment read is the controller’s — including on CI runners, where the environment is whatever the pipeline injected and may include far more than you intended a playbook to see.

`file` and `unvault`

file returns bytes as they are on disk. Point it at a vault-encrypted file and you get the ciphertext, headed $ANSIBLE_VAULT;1.1;AES256 — which is a surprisingly common way to deploy an encrypted blob to a host by accident, since it does not error.

unvault decrypts first, using whatever vault password the run has available:

Read-only / Safereading an encrypted file's plaintext

- ansible.builtin.copy:
  content: "{{ lookup('ansible.builtin.unvault', 'files/api-key.vault') }}"
  dest: /etc/app/api-key
  owner: app
  group: app
  mode: '0400'
  no_log: true

no_log: true is not optional there. Part XXII covers why in full; the short version is that without it the decrypted value appears in -vvv output and in the task result.

`password` — the one with a side effect

Read-only / Safepw.yml

- ansible.builtin.debug:
  msg: "len={{ lookup('ansible.builtin.password', 'pw/demo length=20') | length }}"

- ansible.builtin.debug:
  msg: "same={{ lookup('ansible.builtin.password', 'pw/demo length=20')
                == lookup('ansible.builtin.password', 'pw/demo length=20') }}"

Configuration changeit generates once and remembers— Runs two password lookups against the same path. This writes a file on the controller, which is why it is not a read-only operation.

$ ansible-playbook -i localhost, pw.yml

TASK [password lookup first call] **********************************************
ok: [localhost] => {
  "msg": "len=20"
}

TASK [password lookup second call] *********************************************
ok: [localhost] => {
  "msg": "same=True"
}

The second call returns the same password as the first. It has to, or every run would rotate every credential. And the mechanism by which it remembers is the part people miss:

Read-only / Safethe side effect— Lists the path given to the password lookup. The generated password is stored there in plaintext.

$ ls -la pw/ && wc -c pw/demo

total 4
drwxr-xr-x  2 opsuser opsuser  60 Aug 11 22:45 .
drwxr-xr-x 13 opsuser opsuser 540 Aug 11 22:45 ..
-rw-------  1 opsuser opsuser  21 Aug 11 22:45 demo
21 pw/demo

The path argument is not a key or an identifier. It is a file, and the password is written into it in plaintext, mode 0600, on the controller.

The password lookup creates an unmanaged plaintext credential store

Every consequence of that file is one people discover later:

It is plaintext on the controller. Not vaulted, not encrypted at rest beyond filesystem permissions. 0600 protects it from other users; it does not protect it from a controller backup, a stolen disk, or root.
It is almost certainly inside the repository directory, because the path is relative to the working directory and people write credentials/db_password. One git add -A and it is in history.
Losing it rotates the credential. Delete the file, or run from a different controller, and the next lookup generates a new password — which is then deployed to the host, while the application config elsewhere still has the old one.
Two controllers generate two different passwords for the same logical credential, silently, because neither has the other’s file.

That last point makes it actively unsafe on any estate with more than one controller or a CI runner, which is most of them.

Use a real secret manager, or vault-encrypted variables. If you use password, put the file somewhere deliberate, back it up as a credential, .gitignore it explicitly, and write down that it exists — because the next person will not guess.

`pipe` — the one that is a shell

Read-only / Safepipe runs on the controller, as you— Runs two pipe lookups. The first reports the controller user; the second demonstrates that shell operators are honoured.

$ ansible-playbook -i localhost, pipe.yml

TASK [pipe executes on the controller] *****************************************
ok: [localhost] => {
  "msg": "opsuser"
}

TASK [pipe with a shell metachar] **********************************************
ok: [localhost] => {
  "msg": "one\ntwo"
}

echo one && echo two produced both lines. The && was interpreted, which means this is a shell — not an execve of one binary with arguments. Everything the Linux course says about shell quoting, command substitution and set -o pipefail applies, on the controller, in the security context of the person who ran the playbook.

Merge rights on the repository are shell access on the controller

Follow the chain:

A playbook may contain {{ lookup('pipe', '...') }}.
That string is executed by a shell on the controller when the playbook runs.
The controller runs playbooks as an account with SSH keys to the fleet and access to the vault password.
Therefore anyone who can get a line into a playbook that the controller will run has arbitrary code execution on the controller.

This is not a vulnerability in Ansible. It is what the feature does. But it reframes a governance question that is usually filed under “code style”: who is allowed to approve a change to this repository? If the answer is broader than “people trusted with root on every host in the estate”, the gap is real, and it is not closed by reviewing what the playbook does — it is closed by reviewing what expressions the playbook contains.

Three controls worth having:

Protected branches with required review on any repository the controller runs, which Part XXXVIII covers as CI policy.
A CI grep for pipe used as a lookup name, in both quoting styles, so its introduction is a deliberate, visible event rather than a line in a large diff.
A named list of who may approve. If pipe is permitted at all, the approver list is a root-equivalent privilege list, and should be managed as one.

External secret managers

Every major secret store ships a lookup in its collection — HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, CyberArk, and others. They are all the same shape and all subject to the same three properties, which is more useful to know than any one of their option lists:

1. The controller authenticates, not the target. The credential that unlocks the secret store lives on the controller. That is usually right — the managed node should not be able to read the store — and it concentrates yet more privilege on the controller.

2. The fetch happens during templating. Before the task runs, on every host the expression is evaluated for. A lookup inside a task looped over 300 hosts can be 300 calls to your secret store, and rate limits are real. Fetch once into a variable with set_fact or a play-level vars, and reuse it.

3. no_log is your responsibility. The lookup returns a string. Nothing marks it secret; nothing redacts it downstream. A task whose parameters contain a fetched secret and no no_log: true prints it at -vvv.

Read-only / Safefetch once, use many

- name: Fetch the credential once for the whole play
ansible.builtin.set_fact:
  db_password: "{{ lookup('community.hashi_vault.hashi_vault', 'secret=secret/data/app:password') }}"
run_once: true
delegate_to: localhost
no_log: true

- name: Use it
ansible.builtin.template:
  src: app.conf.j2
  dest: /etc/app/app.conf
  owner: app
  mode: '0400'
no_log: true

Part XXIII covers external secret managers as an architecture. Here the point is narrower: whichever one you use, it is a lookup, so it runs on the controller and everything in this lesson applies.

Why lookups cannot be marked secret automatically

A lookup returns a Python string. By the time it reaches the templating engine there is nothing distinguishing it from any other string — no taint bit, no wrapper type that propagates through filters, no record of where it came from.

That is why no_log is a task attribute rather than something the secret-returning lookup could set for you: the engine would have to track provenance through arbitrary Jinja transformations (| upper, string concatenation, membership in a dict) and there is no mechanism that does.

The practical consequences:

no_log: true belongs on every task whose parameters could contain a fetched secret, and it suppresses the whole task’s output, so you lose the diagnostic detail too. That trade is the right one and it is why Part XXII recommends splitting a task that handles a secret away from tasks you want verbose output from.
Vault-encrypted variables are different. A value loaded from a vaulted file carries its origin through the variable system and Ansible can redact it. A value fetched by a lookup at template time does not.
Callback plugins see task results before you do. A junit or tree callback writing artefacts to disk in CI captures whatever was not suppressed, which is lesson 7’s problem and worth remembering here: no_log is what stops the secret reaching them.

What happens if the secret lookup fails mid-play?

By default, strict: the lookup failure fails the task on the host it was evaluating for, and the play proceeds to other hosts under normal error handling.

That produces the partial-failure shape Part XLIV is about, with a sharper edge than usual. If the store is rate-limiting you, the failures are intermittent and host-dependent: hosts 1–50 configured with the credential, 51–70 failed, 71 onwards configured again as the limit reset. The play ends with a mixed recap and an estate in three different states.

What to do about it:

Fetch once, not per host. run_once: true with delegate_to: localhost makes it one call whose failure is unambiguous and stops the play before anything is changed.
Do not use errors='ignore' on a credential fetch. Lesson 3’s callout applies at its strongest here: an empty secret deploys successfully and fails at 4am in an application log.
any_errors_fatal on the play, so a credential failure on one host stops the rollout rather than producing a partially-configured fleet. Part XXXI covers the trade-offs.

The five

env

file and unvault

password — the one with a side effect

pipe — the one that is a shell

External secret managers

Knowledge check

`env`

`file` and `unvault`

`password` — the one with a side effect

`pipe` — the one that is a shell