Ansible Vault, no_log and external secret lookups

Ansible touches more production hosts per minute than any other tool in most estates, and it does it by rendering templates and shipping modules that carry real credentials. Its two named secret features, Vault and no_log, are both narrower than their names suggest. The documentation is honest about the limits; almost nobody reads that far. This lesson is the part of the manual that decides whether a credential stays inside the run.

Vault protects data at rest, and stops there

The upstream guide states the boundary in capitals: encryption with Ansible Vault only protects data at rest. Once the content is decrypted, which the docs call data in use, the play and plugin authors are responsible for avoiding disclosure. A decrypted vault variable is an ordinary Ansible variable. It templates into files, appears in module arguments, reaches the managed node, and can be printed by anything that prints variables.

That is the correct mental model: Vault solves the problem of a credential sitting in a Git repository. It does not solve the problem of a credential moving through a run.

$ANSIBLE_VAULT;1.1;AES256
$ANSIBLE_VAULT;1.2;AES256;dev

Those two headers tell you what you are looking at. The 1.1 form carries no vault identity; the 1.2 form names one, here dev, which is how a repository can hold material for several environments protected by different passwords. The cipher allowlist contains a single entry, AES256, implemented as AES-CTR with an HMAC-SHA256 authentication code, with the key derived by PBKDF2 at ten thousand iterations.

Ten thousand iterations is not a lot against a modern offline cracking rig. The entire security of a vault file therefore rests on the strength of the vault password, because an attacker who obtains the repository can attack it at their own pace with no rate limit and no lockout. Vault passwords belong in the same class as root passwords, not in the same class as a shared team passphrase.

Two granularities, and the rekey trap

Vault offers exactly two units of encryption. You can encrypt an entire structured data file, or you can encrypt the value of a single variable inside an otherwise readable file. There is no third option: you cannot encrypt a task, a play, or one field of a dictionary.

The two behave differently in three ways that matter. An encrypted file is opaque, so a reviewer cannot see what changed in a diff. An encrypted variable leaves the file legible, which makes review easy. And decryption timing differs: encrypted variables are decrypted on demand when needed, while an encrypted file is decrypted whenever it is loaded or referenced.

That last point carries a footnote worth reading twice. Ansible cannot know whether it needs content from an encrypted file without decrypting the file, so it decrypts all encrypted files referenced in your playbooks and roles. A repository with one vault file per environment, all referenced from a common group_vars tree, hands every password in the tree to every run.

# Variable-level: reviewable file, but this value can never be rekeyed.
ansible-vault encrypt_string

# File-level: opaque in review, and rotatable in one operation.
ansible-vault encrypt group_vars/all/vault.yml
ansible-vault rekey group_vars/all/vault.yml

The last command is the trap. rekey operates on encrypted files. You cannot rekey encrypted variables, which means the convenient, reviewable, variable-level form is the form that makes password rotation a manual re-encryption of every value you ever created. If the vault password is ever disclosed, that difference decides whether recovery takes an hour or a week. Choose file-level encryption wherever rotation matters, and reserve variable-level encryption for values you are prepared to rewrite by hand.

no_log, and the gaps the manual admits

no_log: true strips a task’s arguments and return values from output and logs. Ansible’s own keyword reference describes it as a boolean that controls information disclosure, which is a carefully weaker claim than a security control. Five documented gaps follow from that.

- name: Render the application configuration
  ansible.builtin.template:
    src: app.conf.j2
    dest: /etc/app/app.conf
    owner: app
    group: app
    mode: '0600'
  no_log: true
  diff: false

First, no_log does not affect debugging output, so debugging a playbook in production can print what the flag was meant to hide. Second, it does not prevent disclosure when Ansible itself is debugged through the ANSIBLE_DEBUG environment variable. Third, --diff can reveal sensitive information, which is why the task above sets diff: false explicitly rather than relying on nobody passing the flag. Fourth, a value that appears in a dictionary key name survives, unless the module calls the sanitising helper that strips values from keys. Fifth, redaction works by replacing every string that matches the secret, so a password that is a common word causes every occurrence of that word to be replaced across the output, which mangles logs and reveals that the word is the password.

Two more habits are worth naming. A loop_control label is a readability feature and is explicitly not a means of protecting sensitive data; if a loop iterates over secrets, the task needs no_log. And setting no_log at play level is documented as possible but discouraged, because it makes the play very hard to debug; the recommendation is to apply it to single tasks.

A lookup is a control-node operation

The most common way to remove secrets from a repository is to fetch them from an external manager at run time, through a lookup plugin. The single most important property of a lookup is where it runs.

flowchart LR
    C["Control node\ntemplating engine"] -- "lookup evaluated here" --> M["Secret manager API"]
    M -- "plaintext value" --> C
    C -- "rendered module args" --> H["Managed host\nweb-01"]

Like all templating, lookups execute and are evaluated on the Ansible control machine. The network path to the secret manager, the DNS resolution, the TLS trust decision, the authentication identity and any cloud instance role all belong to the controller. A managed host that cannot reach the secret manager is irrelevant; a controller that cannot reach it fails the whole run. This is also why the controller is the highest-value host in an Ansible estate: it holds, however briefly, every credential every play needs.

- name: Read the application credential on the control node
  ansible.builtin.set_fact:
    db_password: "{{ lookup('amazon.aws.ssm_parameter', '/payments/db_password') }}"
  no_log: true

Use the current fully-qualified names. For HashiCorp Vault and OpenBao the collection is community.hashi_vault, whose lookups include vault_kv2_get for KV version 2 and require the hvac Python library on the controller. For AWS the current names are amazon.aws.secretsmanager_secret and amazon.aws.ssm_parameter; the older aws_secret and aws_ssm names are redirect aliases kept for compatibility.

Two safety details complete the picture. Lookup return values are marked unsafe by default, so Jinja will not re-template them, which protects you from a secret that happens to contain template syntax. And when a looked-up value is passed to a shell, the documentation directs you to apply the quoting filter, because the value came from outside your control.

Production discipline

1. Say what Vault does in one sentence. It protects data at rest; the moment a variable is decrypted the responsibility moves to the play author.
2. Prefer file-level encryption for anything you will rotate. Encrypted variables cannot be rekeyed, and that constraint is discovered at the worst possible time.
3. Pair no_log with diff: false. The flag is documented as able to reveal content, and relying on nobody passing --diff is not a control.
4. Never debug a real inventory with ANSIBLE_DEBUG. It bypasses no_log by design; reproduce on a lab inventory with deliberately fake credentials.
5. Treat the controller as a secret manager. Every lookup runs there, so controller hardening, patching and access control are secret-management work, not general system administration.

Cross-course references

• Ansible for Production Sysadmins - Parts XXI (Secrets Management) and XLVII (Controller Security) cover the day-to-day operation of these features and the controller hardening this lesson treats as a credential boundary.
• Git, CI/CD & GitOps for Infrastructure Engineers - Part XCIV (Incident: Secret Leak) covers the response when a vault password or a decrypted value reaches a repository or a job log.
• Linux for Production Sysadmins - Part LXXII (Secrets) covers the file permissions, shell history and editor artefacts that surround a vault password on the controller itself.

Quiz