The SSH agent, forwarding, and what a bastion can borrow

An agent exists to solve a small problem and creates a larger one. The small problem is that a passphrase-protected key has to be decrypted for every use. The agent decrypts it once, keeps it in memory, and offers a signing service to local processes. The larger problem is that the service is reachable through a socket, and whoever reaches the socket does not need the key.

The agent is a signing service, not a key store

When a client needs to prove possession, it does not read the key file. It sends the agent the exact blob described in the first lesson of this part, together with an identifier for the key it wants used, and receives a signature back. The private key never leaves the agent’s address space, and the client never handles it.

That design has a property worth stating plainly: using the agent and holding the key are indistinguishable from the server’s point of view. A signature produced through an agent is byte-identical to one produced from the file. No audit trail marks it, no field records which process asked, and the agent keeps no history of what it signed. Every control therefore has to sit in front of the request, not behind it.

# Add a key that expires from the agent and requires confirmation
# for every single signature it produces.
ssh-add -t 4h -c ~/.ssh/id_ed25519
ssh-add -l

The -t lifetime bounds exposure in time: the agent forgets the key when it elapses, so a laptop left unlocked stops being a credential after a bounded interval. The -c flag turns each use into a visible event by requiring confirmation, which is the closest thing to an audit trail the agent offers, and it is the single most effective setting on a workstation that connects to production.

The socket is the credential

The agent is addressed through SSH_AUTH_SOCK. Any process that can open that path can ask for signatures with any key currently loaded. The socket’s permissions restrict it to its owner, which is the right default, but two categories of caller sit outside that boundary: root on the same machine, and anything else running as your user, which includes every program you started, every browser extension with shell access and every dependency of a build you ran.

# On any host where an agent socket exists, see who can reach it.
echo "$SSH_AUTH_SOCK"
stat -c '%a %U %G %n' "$SSH_AUTH_SOCK"

Agent forwarding takes this socket and republishes it. With ForwardAgent enabled, or ssh -A on the command line, the remote sshd creates a socket on the remote host, sets SSH_AUTH_SOCK in your shell there, and relays every request it receives back down the existing connection to your local agent. The default for ForwardAgent is no, and that default deserves to be kept.

flowchart LR
    W["Workstation\nagent holds keys"] --> B["Bastion\nforwarded socket"]
    B --> P1["prod-db-01"]
    B --> P2["prod-app-04"]
    A["Attacker with root\non the bastion"] --> B

While your session is open, the socket on the bastion is a working credential for every key your agent holds. An attacker with root there, or with your user id, can list your keys and request signatures for any destination they can reach: the two production hosts in the diagram, the git remote your keys also open, the appliance nobody remembered was on that key. They do not obtain the key, so nothing persists after you disconnect. They do not need to: minutes of your session are enough to install their own authorized_keys entry on everything they touched.

Auditing what an agent exposes right now

The exposure of a forwarded socket is not abstract; it is exactly the set of keys loaded at that moment, and that set is enumerable in one command. Making a habit of looking at it changes behaviour faster than any policy document.

# What is loaded, and what each key can open.
ssh-add -l
ssh-add -L | ssh-keygen -l -f /dev/stdin

The first form prints a fingerprint and comment per key. The second prints the full public keys and pipes them back through fingerprint generation, which is the form you can diff against the entries you find in authorized_keys files across the estate. The question to answer for each line is not “is this key mine” but “what would an hour of this key buy an attacker”. A single key that opens production, the git remote, the network appliances and a personal server is one credential with four unrelated blast radii, and splitting it costs nothing but a Host block.

On a shared machine the same enquiry runs the other way. Every forwarded session creates a socket owned by the connecting user, so the presence of agent sockets on a bastion is a direct measure of how many identities the machine currently borrows. A socket also outlives its usefulness if the session that created it ended abnormally, which is worth knowing because a stale path is indistinguishable from a live one to anyone poking at the filesystem. Counting these on a bastion, and reporting the number, is a cheap and unusually persuasive way to make the case for ProxyJump to a team that believes forwarding is harmless.

Alternatives that give up less

The common reason people forward an agent is to reach a second host through a first one. That requirement does not need forwarding at all. ProxyJump makes the intermediate a transport, not a party.

Host bastion.lab.example
    ForwardAgent no

Host prod-*.lab.example
    ProxyJump bastion.lab.example
    ForwardAgent no

With that configuration the client opens a connection to the bastion, asks it to forward a TCP stream to the target, and then runs a complete, separate SSH session to the target inside that stream. Key exchange, host key verification and your authentication all happen between your workstation and the final host. The bastion sees ciphertext. There is no socket on it, no key list to enumerate and nothing to borrow. Where forwarding genuinely cannot be avoided, three further controls narrow the window.

• Destination constraints. A key can be added to the agent such that the agent will only sign for a nominated path of hosts, enforced by a session-binding extension the client and agent negotiate. A borrowed socket then cannot reach beyond the destinations you authorised.
• Confirmation and lifetime. The -c and -t options from earlier apply to a forwarded socket too, and confirmation is answered on your workstation, not on the bastion. A stolen socket produces a prompt in front of you.
• Hardware-backed keys. A FIDO security key can require a physical touch for every signature. A forwarded socket that reaches such a key is inert unless somebody is standing at your desk. OpenSSH 10.5 also reordered certificate selection to try FIDO keys that need no touch first and PIN or biometric keys last, so that interactive prompts appear only when the alternatives are exhausted.

Production discipline

1. Set ForwardAgent no globally and enable it per host, never the reverse. A default-on setting eventually reaches a machine nobody vetted.
2. Prefer ProxyJump for every multi-hop path. It removes the requirement rather than protecting it, and it makes the final host verify your workstation directly.
3. Load keys with a lifetime and confirmation. Both are one flag each and they convert an invisible capability into a bounded, visible one.
4. Assume a forwarded socket was used, when a bastion is compromised. Rotate the keys that were loaded and review authorized_keys on everything those keys could open.
5. Give production access its own key, loaded only when needed. The blast radius of a borrowed socket is the set of keys in the agent at that moment, and nothing else.

Cross-course references

• Linux for Production Sysadmins - Part LXXII (Secrets) covers keeping credential material off shared machines, which is the general form of the rule against forwarding into a bastion.
• Git, CI/CD & GitOps for Infrastructure Engineers - Part XLII (CI Secrets) covers how a job obtains credentials without holding a long-lived one, the same trade this lesson makes with lifetimes.
• Ansible for Production Sysadmins - Part XLVII (Controller Security) covers protecting the machine that holds automation credentials, which is a permanently loaded agent by another name.

Quiz