Skip to main content
RunBook Academy

Secrets, PKI & CertificatesVII · TLS for OperatorsTLS

Mutual TLS: authenticating the client as well as the server

Advanced⏱ ~24 min🧪 Lab requiredopensslcurl

What you'll learn

  • Describe the three handshake messages mutual TLS adds and where they sit in the flow
  • Explain why a client certificate must assert clientAuth in its extended key usage
  • Enumerate the operational obligations mutual TLS creates beyond the handshake
  • Diagnose a mutual TLS failure from the server side when the client sees only a closed connection

Prerequisites

Practice

Verified against OpenSSL 3.5.x teaching target; 3.0+ minimum · OpenSSH 10.x teaching target; 8.2+ minimum for certificate workflows · OpenBao 2.6.x · Smallstep step-ca 0.30.x · Certbot / Pebble Certbot current release; Pebble 2.10.x ACME test server · Kubernetes (cross-course target) 1.36.x · PostgreSQL 17.x · 2026-08-26

Not yet marked complete on this device.

Ordinary TLS authenticates one party. The client learns which server it reached; the server learns nothing about the client and relies on a token, a password or an API key carried inside the encrypted channel. Mutual TLS moves that identity check down into the handshake, so the connection either belongs to a known peer or never completes. The protocol change is modest. The operational change is not, and teams routinely underestimate the second while enjoying the first.

What mutual TLS adds to the handshake

Three messages appear that a one-way handshake does not carry. The server sends CertificateRequest in its first flight, after EncryptedExtensions and before its own Certificate. The client then answers with a Certificate of its own and a CertificateVerify, before the Finished message it would have sent anyway.

sequenceDiagram
    participant C as Client
    participant S as Server
    C->>S: ClientHello with key_share
    S->>C: ServerHello with key_share
    S->>C: EncryptedExtensions
    S->>C: CertificateRequest (added by mutual TLS)
    S->>C: Certificate then CertificateVerify then Finished
    Note over C,S: client validates the server as usual
    C->>S: Certificate (added by mutual TLS)
    C->>S: CertificateVerify (added by mutual TLS)
    C->>S: Finished
    Note over C,S: server validates the client before any request is read

Everything added sits inside the encrypted portion of the handshake, so a passive observer cannot see which client identity was presented. That is a genuine improvement over sending a bearer token in the first request, and it is one of the strongest arguments for mutual TLS. Note also the ordering: the client authenticates the server first, then presents its own credential. A client never reveals its identity to an unverified peer.

The round trip count does not change. Mutual TLS costs a slightly larger handshake and one extra signature operation on each side, not an extra network round trip.

A client certificate is not a server certificate

The most common cause of a mutual TLS deployment failing on its first day is reusing a server certificate as a client credential. A verifier checks the certificate against an intended purpose, and a certificate that asserts only serverAuth in its extended key usage fails that check:

openssl verify -CAfile root.crt -untrusted srv-ca.crt -purpose sslclient app.crt
error 26 at 0 depth lookup: unsuitable certificate purpose

Issue client credentials with the right extension set from the beginning. The extension file for a client certificate differs from a server one in exactly one line:

basicConstraints=critical,CA:FALSE
keyUsage=critical,digitalSignature
extendedKeyUsage=clientAuth
subjectAltName=DNS:deploy-runner-01.lab.example
openssl req -new -key client.key -sha256 \
  -subj "/CN=deploy-runner-01" -out client.csr
openssl x509 -req -in client.csr -CA client-ca.crt -CAkey client-ca.key \
  -CAcreateserial -sha256 -days 30 -extfile client.ext -out client.crt
openssl verify -CAfile client-ca.crt -purpose sslclient client.crt

Resist the temptation to solve purpose failures by asserting every usage at once. The CA/Browser Forum Baseline Requirements forbid anyExtendedKeyUsage in a publicly trusted TLS subscriber certificate outright, requiring serverAuth and permitting clientAuth. Even in a private hierarchy that is not bound by those rules, a wildcard usage destroys the one signal that stops a stolen server key from being replayed as a client identity.

The moving parts you now have to operate

The handshake change is one message. The operational change is four new obligations, and they are the reason mutual TLS projects stall.

  • A second trust store, on the server. The server already has its own certificate and chain. It now also needs the set of issuers it will accept client certificates from, which is a different file with a different lifecycle. In nginx that is ssl_client_certificate alongside ssl_verify_client on.
  • A credential per client. Every caller needs a key and a certificate delivered to it, stored with correct permissions, and read by the process that makes the connection. That is a secret distribution problem, and it is the same problem you were trying to avoid by not using API keys.
  • Renewal multiplied by the fleet size. One server certificate expiring is an incident. Two hundred client certificates expiring is a schedule, and it needs the same automation, monitoring and reload discipline applied everywhere at once.
  • Revocation that actually has teeth. In a private hierarchy you control both the issuer and every verifier, so a certificate revocation list can genuinely be enforced. That is only true if something loads it and something fails when it is stale. Short lifetimes with fast reissuance remain the more reliable control.

Proving identity is not the same as granting access

A completed mutual TLS handshake establishes exactly one fact: the peer holds the private key for a certificate issued by an authority you configured. It says nothing about what that peer may do. Treating the handshake as an authorisation decision is the design error that turns a well-built mutual TLS deployment into a flat trust domain where any client that can connect can call anything.

The bridge is a deliberate mapping from certificate identity to permission. Take a stable field, normally a subjectAltName entry, and map it to a role in your application or gateway. Two properties make that mapping safe. It must be based on a field the issuing CA controls and a requester cannot choose freely, and it must fail closed for an identity that is authenticated but unmapped.

Diagnosing a failure only the server can see

Mutual TLS inverts the usual troubleshooting asymmetry. In one-way TLS the client holds the diagnosis, because the client is the party performing validation and printing an error. In mutual TLS the server performs the second validation, and the client frequently sees nothing better than a closed connection or a generic alert.

Work from three questions in order. First, did the server actually ask? If CertificateRequest never appears in the flight, client verification is not enabled and no client credential will ever be consulted. Second, did the client actually answer? A client that holds a certificate but was not configured to present it sends an empty Certificate message, which the server treats as no credential at all. Third, did the server accept it? That is a chain, validity and purpose question against the server’s client trust store, and it reproduces offline with the same verifier you would use for a server certificate.

That third step is where the client trust store on the server usually turns out to be the problem: it holds the issuing CA but not the root above it, or it was updated on one node of a pool and not the others, producing a failure that appears intermittent and is not.

Production discipline

  1. Issue client certificates from a separate issuing CA. It gives you a clean trust boundary, lets you revoke or rotate the client hierarchy without touching server certificates, and stops a server credential from ever satisfying a client check.
  2. Keep client certificate lifetimes short and automate renewal before the first one is issued. The fleet only grows, and a manual process that works for five clients fails silently at fifty.
  3. Map identity to permission explicitly, and fail closed. An authenticated but unmapped client must be refused, not defaulted into a permissive role.
  4. Verify enforcement from the wire, not the configuration. Attempt a connection with no client certificate and require it to fail, then repeat that check after every configuration change.
  5. Alert on client certificate expiry by client name. The failing party cannot report its own problem, so the monitoring has to.

Cross-course references

  • Kubernetes for Production Sysadmins - Part LVIII (RBAC) covers binding an authenticated identity to a set of permitted actions, which is precisely the mapping mutual TLS leaves for you to build.
  • Linux for Production Sysadmins - Part XXVI (SSH) covers certificate-based peer authentication in a protocol with a single signing level, a useful contrast with the chain-building described here.
  • Git, CI/CD & GitOps for Infrastructure Engineers - Part XLII (CISecrets) covers delivering credentials to build runners, which is the distribution problem a client certificate fleet inherits.

Quiz

Knowledge check · 4 questions

  1. Q1. A client presents a certificate that asserts only serverAuth in its extended key usage. What happens, and why?

  2. Q2. In a mutual TLS handshake the client validates the server before presenting its own certificate.

  3. Q3. A service is configured for mutual TLS but connections without a client certificate still succeed. Name the two configuration states that produce this and how you would tell them apart.

  4. Q4. Find the fault and decide where the fix belongs.

    A mutual TLS rollout for internal.example.com has been live for a week behind a pool of four gateway nodes. One batch client fails roughly a quarter of its calls with a reset connection and no error text. Other clients are unaffected. The gateway configuration was updated last Tuesday and the client certificate was renewed the same day from a newly added issuing CA.

Passing score: 75%. Answers are checked in this browser.