Who is allowed to run what · Ansible

A controller that four people use through one shared login can answer “what changed” and cannot answer “who changed it”. Those are different questions, and after an incident the second one is the one that decides whether you are investigating a mistake or an intrusion.

This lesson is about the account structure that makes the second question answerable. It is not about trust. Everyone on the team may be entirely trustworthy and the controller will still be unable to tell you which of them ran the play that took production down at 02:40, because the information was never recorded.

The arrangement is familiar: an ansible user on the controller, its home directory holding the repository checkout, the SSH key and the vault password file, and everyone who needs to run automation logs in as it — or sudos to it.

It is popular because it works and because it makes unattended runs trivial. It costs three things.

Attribution. Every run, every command, every Git commit made from that account is attributable to “the ansible user”, which is to say to nobody. Shell history is a shared file that any of them can edit.

Revocation. When someone leaves the team, you cannot remove their access without changing the shared credential, which means changing it for everyone. So it does not get changed, and their access persists for as long as they remember the password — which is the point at which a personnel event becomes a security event.

Least privilege. One account with one set of permissions cannot express “Alex may run against staging” and “Sam may run against production”. Everyone who can use the account can do everything the account can do, which is everything.

Named accounts, without breaking unattended runs

The objection to named accounts is real: a scheduled run belongs to no person, so it needs an account of its own, and once such an account exists people log in as it. The resolution is to make that account usable by automation and not by people.

The structure that works:

Named human accounts, one per engineer, authenticating with their own SSH key or the organisation’s identity system.
A service account for scheduled runs, with no interactive login and no password, whose key is used by the scheduler and by nothing else.
A group that owns the repository checkout, containing the humans and the service account, with the repository group-readable but not group-writable.

The last point is the one Part III’s filesystem layout lesson covers in mechanical detail: a group-writable repository on a shared controller means anyone in that group can edit a playbook that later runs as root everywhere, which is an escalation path that does not look like one.

Read-only / Safe— All four read state only. Run them together: the answer you want is the intersection - who can log in, and of those, who can read the key and the vault password.

Sudo on the controller itself

There are two distinct privileges on a controller and estates routinely conflate them.

Running automation. Executing ansible-playbook against managed nodes. This is enormously powerful — it is root on the fleet — and it does not require root on the controller.

Administering the controller. Patching it, changing its firewall, editing its configuration, reading other users’ files. This requires root on the controller and is a much smaller group.

Conflating them means everyone who runs automation has root on the controller, and therefore can read the vault password file regardless of its mode, take a copy of the private key, and modify the playbooks that run everywhere. The separation is worth the small amount of sudoers work.

# /etc/sudoers.d/automation
# Members of ansible-operators may become the automation service account
# to run scheduled playbooks. They do not get root on this controller.
%ansible-operators ALL=(ansible-run) NOPASSWD: /usr/bin/ansible-playbook

# Controller administration is a separate, smaller group.
%controller-admins ALL=(ALL) ALL

Two honest caveats, because a rule like the first is easy to over-trust.

The ansible-playbook restriction constrains which binary runs, not what it does. Anyone who can run ansible-playbook as the automation account can run any playbook, including one they wrote a minute ago that runs shell: as root on every host. The rule is not a sandbox.

What it does buy is real but narrower: the operator does not get root on the controller, so they cannot read other users’ files, cannot modify the sudoers rule itself, and cannot quietly change the controller’s configuration. It separates fleet privilege from controller privilege, which is worth having, and it should be described that way rather than oversold.

Per-environment separation

The strongest control in this whole part is not on the controller. It is the decision that a credential which works in staging does not work in production.

That decision has three implementations, in increasing order of strength and cost.

Separate keys and separate vault identities, one controller. The production key and the production vault password exist on the controller but are readable only by the production group. Cheap, and it holds up against an ordinary user account compromise. It does not hold up against controller root, because root reads everything.

Separate credentials and separate service accounts, one controller. As above, plus scheduled runs for each environment use a different service account. Better attribution, same ceiling.

Separate controllers. The production credentials do not exist on the machine that touches staging. This is the only arrangement that survives compromise of the staging controller, and it is the reason the answer to “can we use one controller for everything” is usually no on any estate where production matters more than the other environments.

The vault password everyone knows

A vault password that every engineer knows protects the repository against outsiders and against nothing else. Part XXI covers where the password is read from and the mechanisms available; the custody question — who holds it, and what that implies — is lesson 6 of this part.

What belongs here is the accountability consequence, which is separate from the confidentiality one and less often noticed.

Once everyone knows the vault password, decrypting a secret leaves no trace and requires no request. There is no record that a secret was accessed, because access is just reading a file you can already read. When a credential in that vault turns up somewhere it should not be, the set of people who could have taken it is the entire team, and there is no evidence to narrow it.

A password held by fewer people, or fetched from a secret manager that logs the fetch, converts “anyone could have” into “these three people did, at these times”. That is not a smaller confidentiality risk — the same people can still read the same secrets. It is a completely different investigative position.

Under the hood

Ansible itself records almost nothing about who ran what. It has no user model, no audit log of its own, and ansible.cfg settings are advice the controller gives itself.

Everything you get comes from the operating system and from Git:

On the controller — sudo logs each escalation with the invoking user, PAM logs each authentication, and the shell records history per account. All three are per-account, which is why named accounts are the precondition for every other record being meaningful.

On the managed node — sshd logs the key fingerprint that authenticated and sudo logs the escalation. This is the record that survives a compromised controller, because it is written by software the controller does not control. It identifies the key, not the person, which is the strongest practical argument for one key per environment and per purpose: it makes the fingerprint in the target’s log meaningful.

In Git — the commit author, if people commit as themselves. From a shared login they do not, and git config user.name on a shared account is whatever the last person set it to.

Ansible’s own log_path setting records run output on the controller and is genuinely useful for reconstructing what a run did, but it is a file on the controller written by the process being investigated. Treat it as an operational record, not as evidence.

What happens when an engineer with controller access leaves the team?

With named accounts and per-environment credentials, this is a checklist: disable the account, remove it from the groups, remove their key from the controller, and rotate any credential they could read. The last item is the expensive one and its size is exactly the set of credentials their group could read — which you constrained deliberately.

With a shared login, the same event means rotating the shared password, the automation SSH key across the entire fleet, and the vault password, because they knew all three. That is lesson 5 and lesson 6 executed under time pressure, for a routine personnel change.

The practical consequence is that estates with shared logins usually do not do it. The rotation is large enough to need planning, the departure was amicable, and it gets deferred. The access persists, undocumented, until nobody remembers it exists.

That is the real cost of the shared login, and it is not a confidentiality argument. It is that the correct response is expensive enough that it does not happen.

Knowledge check

Why this matters

Access control on a controller is usually discussed as a compliance requirement, and treated as paperwork. It is worth restating as an operational one.

The question an incident actually turns on is not “was the controller secure”. It is “what can we rule out”. A named-account controller with per-environment credentials lets you say: this run came from this account, using this key, which reaches these hosts and no others. Everything outside that set is excluded, and the investigation and the recovery are both bounded.

A shared-login controller lets you say almost nothing, so the response has to assume the maximum: every credential on the host, every host it reaches, everyone who had the login. The technical difference between the two is a handful of accounts and a sudoers file. The difference in what a bad day costs is measured in weeks.