Keys that are only good for automation · Ansible

Part XX covered how Ansible uses SSH keys: ansible_ssh_private_key_file per group, the agent settings that make an unattended run possible, and why separating keys by what they may reach is better than separating them by who is using them.

This lesson is about the key as a credential to be governed: where it comes from, what it may do once it exists, and how you take it away. Those questions are answered on the managed node, not on the controller, which is the entire reason they matter after lesson 1.

One key, one purpose

An automation key should have exactly one job and no human should ever use it.

That sounds like a hygiene rule. It is actually an accountability one. The managed node’s sshd log records the fingerprint of the key that authenticated, not the name of a person. If one key is used both by the scheduler and by three engineers logging in to look at something, that fingerprint means “somebody, doing something”. If it is used only by the scheduled run, the same log line is a fact.

So the estate wants at least these separations:

Automation from human. Engineers authenticate with their own keys or the organisation’s identity system; automation uses keys no person holds.

Environment from environment. A staging key that cannot open a production host is the control that makes a staging compromise survivable. Part XX establishes the mechanism, lesson 3 the organisational case; this is where it becomes a property of the key material itself.

Purpose from purpose, where the purposes differ in blast radius. A key used by a read-only compliance scan is not the key used by the patch run. Constraining the first (see below) is easy and worth doing; the second cannot be meaningfully constrained, which is itself informative.

The single shared root key is the anti-pattern

One key pair, in root’s authorized_keys on every host in the organisation, used by the controller, by two other tools, and by whichever engineers copied it into their own ~/.ssh years ago.

It is common because it is the arrangement you arrive at by never making a decision. Every property in this lesson is absent: no attribution (every host logs the same fingerprint), no environment boundary (staging and production are one blast radius), no restriction possible (root by definition), and no revocation short of touching every host.

The cost is not the risk. The cost is the removal. Once a key is on every host and used by unknown consumers, taking it away requires finding every consumer first — and there is no inventory of them, because nothing recorded who copied it. Teams discover this when they try, break something unrelated, restore the key, and stop.

This is why it belongs in a lesson about building keys rather than one about fixing them. It is far cheaper never to create than to remove.

Constraining the key at the far end

authorized_keys supports options that prefix a key entry and restrict what a session authenticated by it may do. These are enforced by sshd on the managed node.

That sentence is the important one. Everything else in this part is configuration the controller applies to itself, and an attacker holding the controller changes it. authorized_keys options are read by software on the target that does not trust the controller, so they are the only constraint in this course that survives the compromise lesson 1 describes.

The options worth knowing:

Option	Effect
`from="pattern-list"`	The key is only accepted from matching source addresses or hostnames
`command="..."`	Any session using this key runs this command, ignoring what the client asked for
`restrict`	Disables all forwarding, agent forwarding, port forwarding, X11 and PTY allocation
`no-agent-forwarding`	Individually disables agent forwarding
`no-port-forwarding`	Individually disables port forwarding
`expiry-time="YYYYMMDD"`	The key stops being accepted after this time

A restricted entry for a read-only monitoring key looks like this:

restrict,from="192.0.2.10",command="/usr/local/bin/collect-facts" ssh-ed25519 AAAA...REPLACE_ME monitoring@controller

restrict is the right default because it is a deny-by-default: it turns off everything and you re-enable what you need with pty, port-forwarding and similar. Listing individual no-* options instead means every future OpenSSH feature is enabled on your key until somebody adds another no-.

from= is the highest-value restriction for an automation key

command= is powerful and rarely applicable to a general automation key, because Ansible needs to run arbitrary module payloads — that is what a module is. Pinning it to one command breaks everything except a purpose-built single-command key.

from= has no such limitation and applies to every automation key you own. Restricting the key to the controller’s address means a copy of that key, stolen from a backup or a laptop, is useless from anywhere else. It costs one edit per host and it does not constrain what your automation can do at all.

It is not a complete control — an attacker on the controller is at the permitted address, which is exactly the case lesson 1 describes — but it eliminates the entire class of “the key leaked and was used from elsewhere”, which is the more common event.

Read-only / Safe— Both read state only and neither modifies any host. The second runs a read-only module against every host in the pattern, so it opens a session per host.

Passphrase, agent, or unattended

The trade is genuine and the honest framing is that both options are defensible for different runs.

Passphrase plus agent. The key on disk is useless without the passphrase. An engineer unlocks it once per session into ssh-agent, and Ansible uses the agent. This is correct for interactive work and costs nothing operationally.

It does not solve scheduled runs, because there is no human present to type a passphrase at 03:00. Part XX covers what ansible-core 2.21 offers here — the ssh_agent, private_key and private_key_passphrase settings on the ssh connection plugin — which give a real answer to running unattended without a passphrase-free key sitting on disk.

Unattended key with no passphrase. The key on disk is sufficient by itself. This is what most scheduled automation actually uses, and pretending otherwise produces advice nobody follows.

If you are here, be clear about what protects the key: it is filesystem permissions and the security of the controller, and nothing else. That is precisely why the previous three lessons exist, and it is the strongest argument for restricting the key with from= — because the one thing you can add is that a stolen copy does not work elsewhere.

Certificates, and why they are the scalable answer

Everything above manages access by distributing public keys to every host. That approach has a hard ceiling, and the ceiling is revocation.

To remove a key you must edit authorized_keys on every host that has it. Hosts that are down during that run keep the key. There is no central record of which hosts currently trust which keys, other than by asking all of them.

SSH certificates invert this. A certificate authority signs a user certificate; hosts are configured once to trust the CA (TrustedUserCAKeys in sshd_config); and the certificate carries its own constraints — a validity period, a list of principals, and force-command or source-address restrictions embedded in it by the CA rather than by an entry on each host.

The consequences at fleet scale:

Revocation becomes expiry. An automation certificate valid for twenty-four hours does not need to be revoked; it needs to not be renewed. That converts the hardest operation in key management into the absence of an operation.

Adding a host is one configuration line, not a distribution run. The host trusts the CA and therefore trusts every current certificate.

The constraints travel with the credential. Principals and validity are signed into the certificate, so they cannot be edited on one host and forgotten on another.

The cost is a CA to run, protect and make highly available, because a CA outage means nobody can obtain a certificate. That cost is real and it is why certificates are a fleet-scale answer rather than a starting point. On thirty hosts, authorized_keys and a rotation runbook are proportionate. On three thousand, the rotation runbook is the problem and certificates are the answer.

Under the hood

Two mechanical details decide whether restrictions actually work.

from= matches what sshd resolves the client to. With a hostname pattern, that involves a reverse DNS lookup on the managed node, which is a dependency you have just added to every authentication — and an attacker who controls your reverse DNS controls the restriction. Use addresses or CIDR patterns, which are matched directly against the socket, unless you have a specific reason not to.

Options are per key entry, not per file. They prefix the individual line. Adding restrict to the first entry in an authorized_keys file constrains only that key; a second unrestricted entry below it is entirely unaffected. This is the most common way a restriction is believed to be in place and is not — which is the practical reason the audit above matters more than the deployment.

There is a third detail that catches people with command=. Ansible needs to execute arbitrary payloads, so a command= restriction breaks normal module execution outright. Where it genuinely fits — a key whose only job is to trigger one wrapper script — the original client command is available to that script in SSH_ORIGINAL_COMMAND, which is how single-purpose keys are usually built. That is a different design from an Ansible automation key, not a way to constrain one.

What happens if you deploy an authorized_keys restriction that is wrong?

You lose automation access to every host it reached, at once, and the tool you would use to fix it is the tool that just stopped working.

This is the failure that makes people avoid restrictions altogether, and the answer is procedural rather than technical. Apply to one host first and verify with a real run. Apply to a group only after that. Keep a break-glass path that does not depend on the key you are changing — a console, a separate emergency key with its own restrictions, or an out-of-band management interface.

The next lesson makes the same point about rotation, because it is the same problem: any change to how the fleet authenticates you must be verified before the old path is removed. Restrictions are the version of that problem where the old path is removed in the same instant as the new one is applied, which is what makes them sharper than a rotation and worth practising on one host.

Knowledge check

Why this matters

An automation SSH key is usually created in thirty seconds, by someone solving a different problem, and then lives for a decade.

Almost every property that decides how bad a future incident is gets fixed in those thirty seconds: whether the key is shared with humans, whether it spans environments, whether it is root, whether anything constrains it at the far end, and whether anyone will be able to enumerate where it went.

None of those are hard to get right at creation and all of them are expensive to change later — the shared root key is the extreme case, but every one of them has the same shape. The removal costs more than the creation, and the cost grows with time.

That is the argument for treating key creation as a design decision with a written answer, rather than as a step in a runbook nobody revisits.