Skip to main content
RunBook Academy

Secrets, PKI & CertificatesVII · TLS for OperatorsTLS

What TLS actually provides, and what a certificate really does

Intermediate⏱ ~20 minopensslcurl

What you'll learn

  • Distinguish the confidentiality, integrity and server authentication guarantees TLS makes from the guarantees it never made
  • Explain why the certificate public key contributes no bits to the session keys in TLS 1.3
  • Describe how CertificateVerify proves private-key possession for one specific handshake
  • Predict what an attacker does and does not gain from stealing a server private key

Prerequisites

None — start here.

Verified against OpenSSL 3.5.x teaching target; 3.0+ minimum · OpenSSH 10.x teaching target; 8.2+ minimum for certificate workflows · OpenBao 2.6.x · Smallstep step-ca 0.30.x · Certbot / Pebble Certbot current release; Pebble 2.10.x ACME test server · Kubernetes (cross-course target) 1.36.x · PostgreSQL 17.x · 2026-08-26

Not yet marked complete on this device.

TLS gives a client three things and quietly refuses a fourth. Confidentiality, so a passive observer on the path learns nothing about the payload. Integrity, so an active attacker cannot alter a byte undetected. Server authentication, so the client knows which named party answered. What it never gives is a statement about what that party is permitted to do. Almost every TLS misunderstanding an operator carries is a confusion between those first three and a fourth thing the protocol was never asked to supply.

Three guarantees, and the things TLS never promised

The three guarantees are not independent. Confidentiality without authentication is close to worthless, because you would have an encrypted channel to an unknown party. That is why the interesting operational failures are almost always authentication failures, never encryption failures. A browser refusing a connection is telling you it could not establish who it was talking to. It is not telling you the cipher was weak.

  • Confidentiality is symmetric and ephemeral. The bulk of the data is protected by an AEAD algorithm keyed from a secret established fresh for this connection. That secret is discarded when the connection ends.
  • Integrity is not a separate step. In TLS 1.3 every record is protected by an AEAD construction, so decryption and authentication happen together. There is no mode in which the payload decrypts but the tag is unchecked.
  • Server authentication is a naming claim. The server proves it controls a private key, and a certification authority you already trust has asserted that the matching public key belongs to a particular DNS name.
  • Authorisation is absent. A valid certificate for api.example.com says nothing about which API routes the peer may call. That decision belongs to your application.
  • Availability is absent. TLS will happily fail closed. An expired certificate produces an outage, not a downgrade.

One further boundary deserves naming early, because it decides who is actually accountable for an identity. TLS authenticates whichever process terminates the connection, and that is frequently not the process that answers the request. A load balancer, a reverse proxy or a service mesh sidecar holds the private key, completes the handshake, and then opens a second connection to the backend. The client authenticated the terminator. It learned nothing about the hop behind it, and it has no way to tell whether that hop used TLS at all.

So the first question in any design review is where the key lives, because that location is the real edge of the guarantee. If termination happens in a shared load balancer tier, then the certificate and its private key sit in the custody of whoever operates that tier, and a compromise there is a compromise of your service identity even though nothing on your own hosts changed. If termination happens in the application process, the guarantee reaches all the way to the code that answers, and the key inherits every weakness of that deployment pipeline instead.

Where the session keys really come from

RFC 9846, which since July 2026 obsoletes both RFC 8446 and RFC 5246, describes the key schedule as taking exactly two possible input secrets: a pre-shared key, and the asymmetric shared secret produced by the asymmetric key establishment. That establishment is the ephemeral Diffie-Hellman exchange carried in the key_share extensions of ClientHello and ServerHello. The certificate’s public key does not appear anywhere in that list.

You can see the separation in a single connection summary:

HOST=app.lab.example
PORT=443
openssl s_client -connect "$HOST:$PORT" -servername "$HOST" </dev/null
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Protocol: TLSv1.3
Verify return code: 0 (ok)

Read the cipher name carefully. TLS_AES_256_GCM_SHA384 names an AEAD algorithm and a hash, and nothing else. It does not say RSA. It does not say ECDSA. It does not say ECDHE. In TLS 1.3 the key agreement is chosen through supported_groups and key_share, and the certificate’s signature algorithm through signature_algorithms. Those are three independent negotiations, and the cipher suite name covers only one of them.

The certificate’s actual job: a name bound to a public key

flowchart LR
    KS["key_share in ClientHello\nand ServerHello"] --> SS["ephemeral (EC)DHE\nshared secret"]
    SS --> HK["HKDF key schedule"]
    HK --> TK["handshake and application\ntraffic keys"]
    CE["Certificate:\nSAN bound to a public key"] --> CV["CertificateVerify:\nsignature over the transcript"]
    CV --> OK["client accepts the\nserver identity"]
    TK --> OK

Two chains run in parallel and only meet at the end. The left chain produces keys and never consults the certificate. The right chain produces an identity decision and never produces a key. A connection is only usable when both succeed, which is why an identity failure and a key-agreement failure look completely different on the wire even though both end in a dead connection.

What the certificate carries is a CA-attested binding. Inspect one:

CERT=/etc/ssl/certs/app.lab.example.pem
openssl x509 -in "$CERT" -noout -subject -issuer -dates
openssl x509 -in "$CERT" -noout -ext subjectAltName
subject=CN=app.lab.example
issuer=O=RunBook Academy Lab, CN=RunBook Lab Server Issuing CA
notBefore=Aug 26 21:19:00 2026 GMT
notAfter=Nov 24 21:19:00 2026 GMT
X509v3 Subject Alternative Name:
    DNS:app.lab.example, DNS:www.app.lab.example

The subjectAltName entries are the names this certificate speaks for. Under RFC 9525 a client matches the hostname it asked for against those entries and must ignore the Common Name entirely. The certificate also carries the public key that the client will use to check the server’s signature. It carries no capability to encrypt anything on its own.

Why the misconception costs real outages

“The certificate encrypts the traffic” is not a harmless simplification. It produces three specific bad decisions that turn up in incident reviews.

The first is treating expiry as a cryptographic weakening rather than an identity failure. When a certificate lapses, the AEAD is exactly as strong as it was the day before. What has failed is the CA’s willingness to vouch for the name, which is precisely the part you cannot safely wave through. The pressure to “just ignore the warning for a few hours” is a request to disable the only control that failed.

The second is misjudging a key compromise. In TLS 1.3 every key exchange is ephemeral, so an attacker holding the stolen server key cannot decrypt a packet capture taken last month. What they can do is sign a CertificateVerify in a fresh handshake and impersonate the service until the certificate is replaced. That reverses the usual priorities: the retrospective data-disclosure question is closed, and the live impersonation question is urgent.

The third is assuming this was always true. TLS 1.2 offered static RSA suites, named TLS_RSA_WITH_*, in which the client encrypted a secret to the certificate’s public key and the server decrypted it. Under those suites the stolen key did decrypt recorded traffic. RFC 9846 removed them, and RFC 9325 already advised against negotiating them. If a legacy endpoint still allows static RSA, the old retrospective risk is back.

Production discipline

  1. Say what failed, not that TLS failed. In an incident, name the specific guarantee that broke: chain construction, name matching, validity window, or key agreement. “TLS is broken” sends four people down four wrong paths.
  2. Keep the private key on one host, or in one HSM or KMS. The certificate is public by design and the key is not. Every copy you make is a separate rotation event waiting to happen.
  3. Record the certificate serial and public key fingerprint at deploy time. When you later need to prove which key a service is serving, comparing fingerprints is faster and less ambiguous than reading a chain.
  4. Treat any proposal to disable verification as an outage of the authentication guarantee. If someone reaches for it under pressure, the correct response is to fix naming or trust, not to turn off the check.

Cross-course references

  • Linux for Production Sysadmins - Part LXXI (TLS) covers configuring TLS on Linux services, which is where the certificate and key discussed here are actually wired into a daemon.
  • Kubernetes for Production Sysadmins - Part CXIV (TLS) covers in-cluster TLS termination, where the same identity-versus-key distinction decides whether a failure is an Ingress problem or a trust problem.
  • Observability for Production Sysadmins - Part LXIV (TLSMonitoring) covers probing certificate validity from outside the service, turning the expiry described here into a page rather than an outage.

Quiz

Knowledge check · 4 questions

  1. Q1. In a TLS 1.3 session, where do the keys that protect application data come from?

  2. Q2. Stealing a TLS 1.3 server private key lets an attacker impersonate the service on new connections, but does not let them decrypt sessions recorded last month.

  3. Q3. Name the TLS 1.3 handshake message in which a server proves it holds the private key matching its certificate, and state what value that message signs.

  4. Q4. Decide what actually failed, and answer the question the incident manager is asking.

    At 09:12 UTC the payments team reports that calls to api.example.com are failing. Browsers show a certificate error. The incident manager asks whether customer data has been travelling unencrypted since the certificate lapsed at 00:00 UTC, and asks you to disable verification in the client library until a new certificate is issued.

Passing score: 75%. Answers are checked in this browser.