Lab 17: Build an SSH certificate authority

Objective

By the end of this lab you will have replaced authorized_keys with a certificate authority. A user will log in to a server on which their public key appears nowhere at all, and the client will verify the server’s identity without ever being asked to trust a fingerprint.

Two things make that possible, and they are separate systems that happen to use the same tool. TrustedUserCAKeys on the server turns one CA public key into standing permission for every certificate that CA signs. An @cert-authority line in the client’s known_hosts does the mirror image, turning one CA public key into standing trust for every host certificate that CA signs. You will build both, and you will keep their signing keys apart.

You will also meet the two refusals that produce no certificate error whatsoever, which is what makes SSH CA rollouts feel haunted the first time.

Architecture

Two CA key pairs live on your host and never leave it. The server holds only the public half of the user CA. The client holds only the public half of the host CA. Neither side has any per-user or per-host key registered with the other.

flowchart LR
    UCA["user CA private key\nsigns people"] --> UC["alice-cert.pub\nprincipal deploy, 1 hour"]
    HCA["host CA private key\nsigns machines"] --> HC["hostkey-cert.pub\nprincipal sshd.lab.example"]
    UCAP["user_ca.pub in\nTrustedUserCAKeys"] --> SRV["sshd\nAuthorizedKeysFile /dev/null"]
    UC --> SRV
    HC --> SRV
    HCAP["host_ca.pub as\n@cert-authority *.lab.example"] --> CLI["ssh client"]
    SRV --> CLI

Read the diagram as two independent trust decisions made in opposite directions. The server decides whether to believe a person, using the user CA. The client decides whether to believe a machine, using the host CA. Nothing crosses over, and that is deliberate: a CA that can mint host identities must not also be able to mint operator identities.

Requirements

OpenSSH 10.x client, with ssh and ssh-keygen. The principal behaviour described in Task 9 changed in 10.3, so on an older release you will see the same refusal for a different documented reason.
Docker 29.x, able to pull alpine:3.22 and publish a port on 127.0.0.1.
About 60 MB of disk and a working directory under $HOME.
No out-of-band access requirement. Your host’s own sshd, ~/.ssh and firewall are never touched. The only server this lab reconfigures runs inside a container and listens on 127.0.0.1:12222.

Scenario

Your estate has 300 Linux hosts and 40 engineers. Onboarding means writing a public key to 300 authorized_keys files; offboarding means proving you removed it from all 300, including the four that were down for maintenance that week. Last quarter an audit found a departed contractor’s key still present on nine hosts, and nobody could establish when it had last been used.

You have been asked to prototype certificate-based access on disposable infrastructure and to come back with a concrete answer on two questions: what happens when a certificate expires, and what stops a certificate from working everywhere at once.

Tasks

Task 1 — Record the starting state

LAB="$HOME/rbpki-lab-17"
rm -rf "$LAB"
mkdir -p "$LAB/ca" "$LAB/client" "$LAB/server"
cd "$LAB"

{
  echo "containers:"
  docker ps -a --filter name=rbpki- --format '{{.Names}}'
  echo "networks:"
  docker network ls --filter name=rbpki- --format '{{.Name}}'
} > "$LAB/state.pre-lab"
cat "$LAB/state.pre-lab"

Both lists must be empty. A pre-existing rbpki- resource means Cleanup would delete something that is not yours to delete, so rename this lab’s resources first if you find one.

Task 2 — Create two certificate authorities

LAB="$HOME/rbpki-lab-17"
cd "$LAB/ca"

ssh-keygen -t ed25519 -f user_ca -C "RunBook Lab User CA" -N ""
ssh-keygen -t ed25519 -f host_ca -C "RunBook Lab Host CA" -N ""
chmod 600 user_ca host_ca
ls -l

Two key pairs, two roles, no overlap. The private halves stay in $LAB/ca and are the only genuinely sensitive material in this lab; the .pub halves are what you will distribute.

Ed25519 is the right choice for a CA key here. It is small, it is fast, and ssh -Q key on any 10.x build lists it first. Note that CASignatureAlgorithms on the server controls which algorithms a CA is allowed to have signed with, and a certificate signed with an algorithm outside that list is not accepted for authentication at all.

LAB="$HOME/rbpki-lab-17"
{
  echo "role: signs USER certificates - installed as TrustedUserCAKeys on servers"
  ssh-keygen -l -f "$LAB/ca/user_ca.pub"
  echo "role: signs HOST certificates - installed as @cert-authority in client known_hosts"
  ssh-keygen -l -f "$LAB/ca/host_ca.pub"
} > "$LAB/ca-inventory.txt"
cat "$LAB/ca-inventory.txt"

The user CA private key is the most valuable object in the estate

Anyone holding user_ca can mint a certificate for any principal your fleet accepts, with any validity window, and it will be accepted everywhere the CA is trusted. There is no revocation list to consult at issuance time and no second approval step. The key alone is the authority.

In production it belongs in an HSM or a secrets manager that performs the signing rather than handing the key out, ssh-keygen supports both routes: -D signs with a key in a PKCS#11 token and -U signs with a key held in an agent, so the private half never has to be a file on the signing machine. The lab keeps it in a file because the lab deletes it in twenty minutes.

Keeping the host CA separate is the same argument applied twice. Compromise of the host CA lets an attacker impersonate servers; compromise of the user CA lets them become an operator. One key that can do both turns a single leak into a total loss.

Task 3 — Issue a user certificate

LAB="$HOME/rbpki-lab-17"
cd "$LAB/ca"
ssh-keygen -t ed25519 -f "$LAB/client/alice" -C "alice@lab" -N ""
cp "$LAB/client/alice.pub" .

Now sign it. Four flags carry all the policy.

Configuration changelab host - signing a user key with the user CA— The confirmation restates every decision the certificate now encodes.

$ ssh-keygen -s user_ca -I "alice@runbook-lab" -n deploy -V -5m:+1h -z 1001 alice.pub

Signed user key alice-cert.pub: id "alice@runbook-lab" serial 1001 for deploy valid from 2026-08-26T21:16:24 to 2026-08-26T22:21:24

Illustrative output

-I is the key identity, a free-text label the server logs on every use. This is your audit trail, so put something in it that maps back to a person and a request.
-n deploy is the principal. For a user certificate this is the Unix account name the holder may log in as. Wildcards are not supported in user certificate principals as of 10.3; they are a host-certificate feature only.
-V -5m:+1h is the validity window, starting five minutes ago to absorb clock skew and ending in one hour.
-z 1001 is the serial number, which is what a revocation list will refer to. The default is zero, and zero cannot be revoked by serial.

The output file is derived, not chosen: signing alice.pub writes alice-cert.pub beside it. Read it back.

Read-only / Safelab host - every field the certificate actually carries— Nothing here is fetched at authentication time; it is all inside the signed blob.

$ ssh-keygen -L -f alice-cert.pub

alice-cert.pub:
      Type: ssh-ed25519-cert-v01@openssh.com user certificate
      Public key: ED25519-CERT SHA256:/hOjaMxlXaTaWekzkFCIxR4PRU/c7eLkooB565Y7X6I
      Signing CA: ED25519 SHA256:6x39cg8OAp7PZgisHreCLRO7g0vp6s0VZIw+DcTgKtE (using ssh-ed25519)
      Key ID: "alice@runbook-lab"
      Serial: 1001
      Valid: from 2026-08-26T21:16:24 to 2026-08-26T22:21:24
      Principals:
              deploy
      Critical Options: (none)
      Extensions:
              permit-X11-forwarding
              permit-agent-forwarding
              permit-port-forwarding
              permit-pty
              permit-user-rc

Illustrative output

Your fingerprints and timestamps will differ; the structure will not. Save this listing, it is a deliverable.

The five extensions are the defaults. Each has a no- form you can pass to -O to switch it off, and the man page marks all four forwarding and pty items as permitted by default. Critical options are the opposite kind of field: force-command, source-address and verify-required are the defined set, and a server that does not understand a critical option must refuse the certificate rather than ignore it.

What an SSH certificate is, and what it deliberately is not

An OpenSSH certificate is a single signed blob with a fixed field order: a nonce, the public key material, a 64-bit serial, the certificate role (user or host), the key identity string, the principals list, the valid-after and valid-before times, critical options, extensions, a reserved field, the CA’s signature key and the signature itself. The nonce comes first, before the key material, specifically so that an attacker cannot control the prefix of the data being hashed.

The ssh-keygen manual states the comparison plainly: OpenSSH certificates are a different, and much simpler, format to the X.509 certificates used in ssl. There is exactly one signature and exactly one level. There are no intermediates, no path building, no extension OIDs, no CRL distribution point and no OCSP responder. Revocation is a key revocation list or a plain list of keys that the server reads locally, which is the subject of Lab 18.

The normative specification is draft-ietf-sshm-cert, an active IETF Internet-Draft rather than an RFC. The older PROTOCOL.certkeys file that most tutorials still cite was deleted upstream in May 2025 and is absent from every release from 10.1 onward, so a document that quotes it is describing a file that no longer ships.

Task 4 — The `-V` trap

Sign a throwaway key with no -V at all, then read the validity window back.

LAB="$HOME/rbpki-lab-17"
cd "$LAB/ca"
ssh-keygen -t ed25519 -f eternal -C "eternal-probe" -N ""
ssh-keygen -s user_ca -I "eternal-probe" -n deploy -z 9999 eternal.pub
ssh-keygen -L -f eternal-cert.pub | grep -A1 Valid

Read the Valid: line you just produced before continuing. The manual is unambiguous about what you will find: by default, certificates are valid from the Unix Epoch to the distant future. An unqualified signing operation does not produce a short-lived credential, it produces a permanent one.

LAB="$HOME/rbpki-lab-17"
rm -f "$LAB/ca/eternal" "$LAB/ca/eternal.pub" "$LAB/ca/eternal-cert.pub"

-V accepts absolute and relative forms, and the manual documents both: -4w:+4w, +52w1d, -1d:20110101, 20100101123000Z:20110101123000Z, and raw epoch seconds as 0x1:0x2000000000. A start time slightly in the past is normal practice and is why this lab uses -5m rather than always.

Task 5 — Issue the host certificate

LAB="$HOME/rbpki-lab-17"
cd "$LAB/server"
ssh-keygen -t ed25519 -f hostkey -N ""
chmod 600 hostkey
cp hostkey.pub "$LAB/ca/"
cd "$LAB/ca"

Configuration changelab host - -h is what makes this a host certificate rather than a user one— The principal of a host certificate is a hostname, not an account name.

$ ssh-keygen -s host_ca -I "sshd.lab.example" -h -n sshd.lab.example -V -5m:+52w hostkey.pub

Signed host key hostkey-cert.pub: id "sshd.lab.example" serial 0 for sshd.lab.example valid from 2026-08-26T21:16:24 to 2027-08-25T21:21:24

Illustrative output

Note serial 0: no -z was given, and zero is the default. That is acceptable for a host certificate you intend to replace by reissuing, and it is a problem the moment you need to revoke it by serial, because serial numbers used in a revocation list explicitly exclude zero.

Read it back and compare against the user certificate.

Read-only / Safelab host - a host certificate carries no extensions at all— The permit-* extensions are user-certificate concepts; no standard options are valid for host keys.

$ ssh-keygen -L -f hostkey-cert.pub

hostkey-cert.pub:
      Type: ssh-ed25519-cert-v01@openssh.com host certificate
      Public key: ED25519-CERT SHA256:FVtUBGJzmer6HUh/WV7JSu/KTKqGBu7NakyUTr2/n7E
      Signing CA: ED25519 SHA256:KS2EMGtUNk0IWh7aTMPH/7iu4rexBxku9af6JK42AqU (using ssh-ed25519)
      Key ID: "sshd.lab.example"
      Serial: 0
      Valid: from 2026-08-26T21:16:24 to 2027-08-25T21:21:24
      Principals:
              sshd.lab.example
      Critical Options: (none)
      Extensions: (none)

Illustrative output

Extensions: (none) is not an omission. The manual states that at present no standard options are valid for host keys, so the field is empty by design. A 52-week host certificate is also a deliberate contrast: host identities change rarely, operator access changes constantly, and the two deserve different lifetimes even though the same tool issues both.

LAB="$HOME/rbpki-lab-17"
ssh-keygen -L -f "$LAB/ca/alice-cert.pub" > "$LAB/user-cert-decoded.txt"
ssh-keygen -L -f "$LAB/ca/hostkey-cert.pub" > "$LAB/host-cert-decoded.txt"
cp "$LAB/ca/hostkey-cert.pub" "$LAB/server/"
cp "$LAB/ca/user_ca.pub" "$LAB/server/"

Task 6 — Configure an sshd that trusts the CA

LAB="$HOME/rbpki-lab-17"

cat > "$LAB/server/sshd_config" <<'SSHDCONF'
Port 22
HostKey /tmp/hostkey
HostCertificate /tmp/hostkey-cert.pub
TrustedUserCAKeys /tmp/user_ca.pub
PubkeyAuthentication yes
PasswordAuthentication no
AuthorizedKeysFile /dev/null
LogLevel VERBOSE
PidFile /tmp/sshd.pid
SSHDCONF
cp "$LAB/server/sshd_config" "$LAB/sshd-ca-config.txt"

cat > "$LAB/server/start.sh" <<'START'
#!/bin/sh
set -e
apk add --no-cache openssh-server >/dev/null
adduser -D deploy
cp /etc/ssh/lab/hostkey /tmp/hostkey
chmod 600 /tmp/hostkey
cp /etc/ssh/lab/hostkey-cert.pub /tmp/hostkey-cert.pub
cp /etc/ssh/lab/user_ca.pub /tmp/user_ca.pub
cp /etc/ssh/lab/sshd_config /tmp/sshd_config
exec /usr/sbin/sshd -D -f /tmp/sshd_config -e
START

docker network create rbpki-net-17
docker run -d --name rbpki-ssh17 --network rbpki-net-17 \
  --network-alias sshd.lab.example \
  -p 127.0.0.1:12222:22 \
  -v "$LAB/server:/etc/ssh/lab:ro" \
  alpine:3.22 sh /etc/ssh/lab/start.sh
sleep 8
docker ps --filter name=rbpki-ssh17 --format '{{.Names}} {{.Status}}'

Four directives do the work. HostKey and HostCertificate must name a matching pair, because the certificate’s public key has to correspond to a private host key the daemon already loaded. TrustedUserCAKeys is the whole authorisation model: any certificate signed by a key listed in that file may authenticate as any user named in its principals list.

AuthorizedKeysFile /dev/null is the proof. With no authorized_keys anywhere, a successful login cannot possibly have come from a registered key.

Notice what is absent: there is no RevokedKeys line. That is not an oversight. If RevokedKeys names a file that is not readable, public-key authentication is refused for every user on the host, which is an outage produced by a typo. Lab 18 adds it deliberately, starting from an empty revocation list.

Task 7 — Teach the client to trust the host CA

LAB="$HOME/rbpki-lab-17"
{
  printf '@cert-authority *.lab.example '
  cat "$LAB/ca/host_ca.pub"
} > "$LAB/client/known_hosts"
cut -c1-70 "$LAB/client/known_hosts"

A known_hosts line is marker, hostname pattern, key type, base64 key and comment. The @cert-authority marker changes the meaning of the whole line: instead of “this is the key of this host”, it says “this is a CA whose host certificates I will accept for any host matching this pattern”. One line replaces every fingerprint you would otherwise accumulate.

Wildcards are supported in host certificate principals, which is why *.lab.example works here and why the equivalent would not work for a user certificate.

Only one marker may appear on a line. The other marker is @revoked, and a key marked revoked is never accepted, including as a certificate authority.

Task 8 — Authenticate on the certificate alone

The client picks up alice-cert.pub automatically because it sits beside the private key named by -i. Copy it into place and connect.

LAB="$HOME/rbpki-lab-17"
cp "$LAB/ca/alice-cert.pub" "$LAB/client/"

ssh -i "$LAB/client/alice" \
    -o "UserKnownHostsFile=$LAB/client/known_hosts" \
    -o StrictHostKeyChecking=yes \
    -o IdentitiesOnly=yes \
    -o BatchMode=yes \
    -o HostKeyAlias=sshd.lab.example \
    -p 12222 deploy@127.0.0.1 id

HostKeyAlias tells the client to match host identity against sshd.lab.example rather than against 127.0.0.1, which is what lets a loopback-published container present a certificate for its real name.

The first attempt fails.

Read-only / Safeclient side - a refusal with no certificate error anywhere in it— The account is the problem, not the credential, and nothing here says so.

$ ssh -i "$LAB/client/alice" \
  -o "UserKnownHostsFile=$LAB/client/known_hosts" \
  -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes \
  -o HostKeyAlias=sshd.lab.example -p 12222 deploy@127.0.0.1 id

deploy@127.0.0.1: Permission denied (publickey,keyboard-interactive).

Illustrative output

Check the server before changing anything.

docker logs rbpki-ssh17 2>&1 | grep -Ei 'cert|principal|expired|revoked|Accepted|Failed'

The diagnostic signal here is an absence. There is no Certificate invalid: line, because the certificate was never the problem. adduser -D on Alpine creates the deploy account with no password, which BusyBox records as a locked account in /etc/shadow, and sshd will not complete public-key authentication for a locked account.

docker exec rbpki-ssh17 passwd -u deploy

LAB="$HOME/rbpki-lab-17"
ssh -i "$LAB/client/alice" \
    -o "UserKnownHostsFile=$LAB/client/known_hosts" \
    -o StrictHostKeyChecking=yes \
    -o IdentitiesOnly=yes \
    -o BatchMode=yes \
    -o HostKeyAlias=sshd.lab.example \
    -p 12222 deploy@127.0.0.1 id

Now run it once more with -v and read what the client reports about both directions of trust.

Read-only / Safeclient side - both trust decisions, visible in one session— The host was verified by certificate, and the user was offered by certificate.

$ ssh -v -i "$LAB/client/alice" \
  -o "UserKnownHostsFile=$LAB/client/known_hosts" \
  -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes \
  -o HostKeyAlias=sshd.lab.example -p 12222 deploy@127.0.0.1 id

debug1: Host 'sshd.lab.example' is known and matches the ED25519-CERT host certificate.
debug1: Offering public key: alice ED25519-CERT SHA256:... explicit
debug1: Server accepts key: alice ED25519-CERT SHA256:... explicit
Authenticated to 127.0.0.1 ([127.0.0.1]:12222) using "publickey".

Illustrative output

There was no first-contact prompt and no fingerprint to compare by eye, because the @cert-authority line already answered that question. The server’s side of the same event names the CA and the key identity.

Read-only / Safeserver side - the audit line a certificate deployment exists to produce— Key identity, serial and issuing CA, all in the log, with no authorized_keys involved.

$ docker logs rbpki-ssh17 2>&1 | grep -E 'Accepted|authorized keys'

Accepted certificate ID "alice@runbook-lab" (serial 1001) signed by ED25519 CA SHA256:6x39cg8... via /tmp/user_ca.pub
Accepted publickey for deploy from 172.25.0.1 port 51328 ssh2: ED25519-CERT SHA256:... ID alice@runbook-lab (serial 1001) CA ED25519 SHA256:...
User 'deploy' authorized keys '/dev/null' is not a regular file

Illustrative output

That third line is benign and appears even on success. It is AuthorizedKeysFile /dev/null doing exactly what you asked, and it is worth recognising so that nobody spends an afternoon chasing it.

Task 9 — Prove the principal is load-bearing

Issue a certificate with no -n at all, and try again.

LAB="$HOME/rbpki-lab-17"
cd "$LAB/ca"
ssh-keygen -s user_ca -I "alice@no-principals" -V -5m:+1h -z 1010 alice.pub
ssh-keygen -L -f alice-cert.pub | grep -A1 Principals
cp alice-cert.pub "$LAB/client/"

ssh -i "$LAB/client/alice" \
    -o "UserKnownHostsFile=$LAB/client/known_hosts" \
    -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes \
    -o HostKeyAlias=sshd.lab.example \
    -p 12222 deploy@127.0.0.1 id

The login is refused. Two separate rules produce that result, and conflating them is how the internet got this wrong.

The first rule has always applied on this path. The manual for TrustedUserCAKeys states that certificates that lack a list of principals will not be permitted for authentication using TrustedUserCAKeys. A principal-less certificate has never worked here.

The second rule is newer and broader. Before OpenSSH 10.3, an empty principals section was treated as a wildcard matching any principal on the authorized_keys principals="" path. Since 10.3 an empty principals section matches nothing, anywhere. Any tutorial, vendor guide or forum answer that tells you to omit -n for a certificate that works everywhere is describing behaviour that has been removed.

Restore the working certificate before moving on.

LAB="$HOME/rbpki-lab-17"
cd "$LAB/ca"
ssh-keygen -s user_ca -I "alice@runbook-lab" -n deploy -V -5m:+1h -z 1001 alice.pub
cp alice-cert.pub "$LAB/client/"
ssh -i "$LAB/client/alice" \
    -o "UserKnownHostsFile=$LAB/client/known_hosts" \
    -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes \
    -o HostKeyAlias=sshd.lab.example -p 12222 deploy@127.0.0.1 hostname

Task 10 — Capture the deliverables

LAB="$HOME/rbpki-lab-17"
cd "$LAB"

ssh-keygen -L -f "$LAB/ca/alice-cert.pub" > "$LAB/user-cert-decoded.txt"
ssh-keygen -L -f "$LAB/ca/hostkey-cert.pub" > "$LAB/host-cert-decoded.txt"
docker logs rbpki-ssh17 2>&1 \
  | grep -Ei 'cert|principal|Accepted|Failed|authorized keys' \
  > "$LAB/cert-auth-evidence.log"

ls -l ca-inventory.txt user-cert-decoded.txt host-cert-decoded.txt \
      sshd-ca-config.txt cert-auth-evidence.log

Validation

The final ssh ... hostname in Task 9 exits 0 and prints the container hostname. A Permission denied at this point means the restored certificate was not copied into $LAB/client.
docker exec rbpki-ssh17 cat /home/deploy/.ssh/authorized_keys fails with “No such file or directory”. If it succeeds, an authorized_keys file exists and the login you just performed did not prove what you think it proved.
grep -c 'Accepted certificate ID' cert-auth-evidence.log is at least 1, and each such line names the key identity you passed to -I. A count of zero means authentication succeeded by some other route.
ssh-keygen -L -f "$LAB/ca/alice-cert.pub" shows a Valid: line with two timestamps roughly 65 minutes apart, Serial: 1001, and deploy under Principals:. A Valid: forever here means a -V was dropped somewhere.
grep -c '@cert-authority' "$LAB/client/known_hosts" returns 1, and grep -c 'sshd.lab.example ssh-ed25519' returns 0, proving no plain host key was ever added by first contact.
The five deliverables exist and are non-empty.

Expected Outcome

$HOME/rbpki-lab-17/
├── state.pre-lab
├── ca-inventory.txt
├── user-cert-decoded.txt
├── host-cert-decoded.txt
├── sshd-ca-config.txt
├── cert-auth-evidence.log
├── ca/
│   ├── user_ca, user_ca.pub
│   ├── host_ca, host_ca.pub
│   ├── alice.pub, alice-cert.pub
│   └── hostkey.pub, hostkey-cert.pub
├── client/
│   ├── alice, alice.pub, alice-cert.pub
│   └── known_hosts
└── server/
    ├── hostkey, hostkey.pub, hostkey-cert.pub
    ├── user_ca.pub
    ├── sshd_config
    └── start.sh

You can now answer the two questions the scenario asked for. A certificate stops working the moment its valid before time passes, with no fleet-wide action and no file to remove, which is what removes offboarding from the critical path. And a certificate is confined by its principals list, which is the only thing standing between “this person may log in as deploy on hosts trusting this CA” and “this person may log in as anyone”.

Troubleshooting

Host key verification failed. The client did not accept the host certificate. Confirm the known_hosts line begins with @cert-authority, that its hostname pattern matches the value you passed to HostKeyAlias, and that the key on that line is host_ca.pub and not hostkey.pub. Pasting the host key instead of the host CA key is the usual slip.

Every login is refused and the server log has no Certificate invalid: line. The account is locked. Run docker exec rbpki-ssh17 passwd -u deploy. This is Task 8, and it will catch you again if you rebuild the container.

The server log says the host certificate does not match the host key. HostKey and HostCertificate name a mismatched pair. Reissue the host certificate from the same hostkey.pub that corresponds to the private hostkey the daemon loads, then restart the container.

sshd exits at start-up. Run docker logs rbpki-ssh17. The common causes are a TrustedUserCAKeys path that does not exist inside the container, and a host key whose mode is not 0600. Both are copied into /tmp by the start script precisely so the modes are under the daemon’s control rather than the bind mount’s.

Cleanup

LAB="$HOME/rbpki-lab-17"

# 1. Stop and forget the container and its network.
docker rm -f rbpki-ssh17
docker network rm rbpki-net-17

# 2. Compare against the Task 1 capture before deleting the evidence.
cat "$LAB/state.pre-lab"
docker ps -a --filter name=rbpki- --format '{{.Names}}'
docker network ls --filter name=rbpki- --format '{{.Name}}'

# 3. Remove the lab directory, both CA private keys included.
rm -rf "$LAB"

To confirm restoration, run docker ps -a --filter name=rbpki- and docker network ls --filter name=rbpki-: both must print only their header row, matching the empty lists in state.pre-lab. Then run test -d "$HOME/rbpki-lab-17" && echo "still present" || echo "removed". Your own ~/.ssh/known_hosts was never written to, because every client invocation named a lab-local file.

Production notes

Signing is a service, not a command. Nothing above stops an operator from issuing themselves a 52-week certificate for principal root. In production the CA key sits behind something that authenticates the requester, enforces the maximum validity, chooses the principals from the requester’s group membership, and writes an issuance record. ssh-keygen -D and -U exist so the signing key can stay in a token or an agent while that service runs.
Short windows replace revocation. A one-hour certificate needs no revocation list because it expires before a revocation could propagate. Lab 18 builds the revocation machinery for the cases where an hour is still too long, but the primary control is the validity window.
Host certificates end fingerprint fatigue. The reason people type yes at the first-contact prompt without reading is that they do it forty times a week. One @cert-authority line makes the prompt disappear for the whole estate, which converts an ignored security control into an enforced one.
Plan the CA rotation before you need it. Trust is one public key in one file on every host, so rotation means distributing a second CA key, running both, reissuing, and removing the first. TrustedUserCAKeys accepts several keys, one per line, which is what makes that overlap possible.
Do not add RevokedKeys casually. An unreadable RevokedKeys file refuses public-key authentication for every user, and the client-side symptom is the same Permission denied as everything else.

What You Learned

A certificate authority replaces distribution with trust. The server holds one CA public key and no user keys; the client holds one CA public key and no host keys. Onboarding and offboarding stop being fleet-wide file edits.
-V is not optional. Without it, ssh-keygen -s signs a certificate valid from the Unix epoch to the distant future. There is no server-side cap to catch the mistake, so the enforcement has to live in your issuance wrapper.
Empty principals match nothing. Since OpenSSH 10.3 an empty principals section is never a wildcard, and TrustedUserCAKeys never accepted principal-less certificates at all. Without AuthorizedPrincipalsFile, the account being logged into must itself appear in the principals list.
User and host CAs are different jobs. One key that can mint both operator identities and server identities converts a single key compromise into a complete estate compromise.
A refusal with no Certificate invalid: line is not a certificate problem. Account state, permissions and Match blocks all produce the identical client message, and a locked freshly created account is the most common of them.

Objectives

Objective

Architecture

Requirements

Scenario

Tasks

Task 1 — Record the starting state

Task 2 — Create two certificate authorities

Task 3 — Issue a user certificate

Task 4 — The `-V` trap

Task 5 — Issue the host certificate

Task 6 — Configure an sshd that trusts the CA

Task 7 — Teach the client to trust the host CA

Task 8 — Authenticate on the certificate alone

Task 9 — Prove the principal is load-bearing

Task 10 — Capture the deliverables

Validation

Expected Outcome

Troubleshooting

Cleanup

Production notes

What You Learned

Deliverables

Verification status

Objectives

Objective

Architecture

Requirements

Scenario

Tasks

Task 1 — Record the starting state

Task 2 — Create two certificate authorities

Task 3 — Issue a user certificate

Task 4 — The -V trap

Task 5 — Issue the host certificate

Task 6 — Configure an sshd that trusts the CA

Task 7 — Teach the client to trust the host CA

Task 8 — Authenticate on the certificate alone

Task 9 — Prove the principal is load-bearing

Task 10 — Capture the deliverables

Validation

Expected Outcome

Troubleshooting

Cleanup

Production notes

What You Learned

Deliverables

Verification status

Task 4 — The `-V` trap