TLS and PKI concepts - the foundation of encrypted communication

TLS (Transport Layer Security) provides encrypted communication. PKI (Public Key Infrastructure) provides the trust model. This lesson covers the concepts and the production discipline.

TLS handshake

When a client connects to a server over TLS:

1. Client sends ClientHello (supported ciphers, random).
2. Server sends ServerHello (chosen cipher, certificate).
3. Client verifies the server’s certificate against its trust store.
4. Client and server exchange keys (key exchange).
5. Encrypted communication begins.

The certificate is the trust anchor: the client trusts the server because the certificate is signed by a CA the client trusts.

Certificate structure

A certificate contains:

• Subject: the entity (host, person, service).
• Issuer: the CA that signed the certificate.
• Public key: for encryption and verification.
• Validity period: notBefore and notAfter.
• Serial number: unique identifier.
• Extensions: subjectAltName, basicConstraints, keyUsage, and the rest of the X.509 v3 set.
• Signature: the CA’s signature.

X.509 is the standard certificate format.

The hostname lives in the subjectAltName

The Subject Common Name is not how a client decides that a certificate belongs to a host. That role belongs to the subjectAltName extension. RFC 2818 deprecated the CN fallback in 2000 and RFC 9525 removed it. Chrome has required a SAN since version 58, and Go has rejected CN-only certificates since 1.15, which covers Prometheus, Consul, Vault, Docker and most Kubernetes components.

A certificate with no subjectAltName is not a valid server certificate, whatever its CN says. OpenSSL still accepts it, which is exactly why the mistake survives testing and surfaces at cutover.

Public Key Infrastructure (PKI)

PKI is the system of trust:

• Root CA: the trust anchor. Signs intermediate CAs.
• Intermediate CA: signs end-entity certificates.
• End-entity certificate: signs for a host, person, or service.

A chain of certificates establishes the trust path. The client trusts the root CA; the root CA trusts the intermediate; the intermediate trusts the end-entity.

Certificate lifecycle

Every certificate has a lifecycle:

1. Generate: create a key pair and CSR.
2. Sign: send CSR to CA; CA returns the certificate.
3. Deploy: install certificate and key on the server.
4. Use: clients connect and verify.
5. Renew: before expiration, request a new certificate.
6. Revoke (if compromised): mark the certificate as no longer trusted.

A certificate that expires without renewal causes service disruption. The discipline: monitor and renew before expiration.

Public CA vs private CA

• Public CA (Let’s Encrypt, DigiCert): trusted by all browsers and clients. For public-facing services.
• Private CA (internal): trusted only by the organisation. For internal services.

For internal services (e.g. a private API), a private CA is appropriate. For public services, a public CA.

Set up a private CA with openssl

This is a two-level CA, suitable for a lab or a small internal service. Run it in a scratch directory; it writes only to the current directory and installs nothing.

# Generate root CA key and certificate
openssl genrsa -out root-ca.key 4096
chmod 600 root-ca.key
openssl req -x509 -new -nodes -key root-ca.key \
    -sha256 -days 3650 -out root-ca.crt \
    -subj "/C=US/ST=CA/L=SF/O=MyOrg/CN=MyOrg Root CA" \
    -addext "basicConstraints=critical,CA:TRUE" \
    -addext "keyUsage=critical,keyCertSign,cRLSign" \
    -addext "subjectKeyIdentifier=hash"

# Generate the server key and CSR, naming the host in the SAN
openssl genrsa -out server.key 2048
chmod 600 server.key
openssl req -new -key server.key -out server.csr \
    -subj "/C=US/ST=CA/L=SF/O=MyOrg/CN=server.example.com" \
    -addext "subjectAltName=DNS:server.example.com"

# The CA decides the extensions it issues
cat > server-ext.cnf <<'EOF'
subjectAltName = DNS:server.example.com
basicConstraints = critical,CA:FALSE
keyUsage = critical,digitalSignature,keyEncipherment
extendedKeyUsage = serverAuth
EOF

openssl x509 -req -in server.csr -CA root-ca.crt \
    -CAkey root-ca.key -CAcreateserial \
    -out server.crt -days 365 -sha256 \
    -extfile server-ext.cnf

-extfile is not optional. openssl x509 -req adds no X.509 v3 extensions and silently drops the ones requested in the CSR. Without it you get a CN-only certificate that signs cleanly and then fails on every modern client.

Confirm before deploying:

openssl x509 -in server.crt -noout -ext subjectAltName
openssl verify -CAfile root-ca.crt \
    -verify_hostname server.example.com -purpose sslserver \
    server.crt
# server.crt: OK

Distribute root-ca.crt to clients (the trust store). Deploy server.crt and server.key on the server. Remove the scratch directory when you are done; nothing here needs to be undone on the host itself.

Knowledge check