Extensions that decide behaviour: basicConstraints, keyUsage and extendedKeyUsage

Three extensions decide almost everything about how a certificate is allowed to behave. basicConstraints says whether the holder may sign other certificates, keyUsage says which cryptographic operations the key may perform, and extendedKeyUsage says which application roles the certificate may be presented for. A verifier reads all three before it will finish a handshake, and when one of them is wrong the failure message names the symptom rather than the cause.

What critical actually obliges a verifier to do

Criticality is a single boolean attached to each extension, and it changes the contract between the issuer and every verifier that will ever read the certificate. RFC 5280 §4.2 puts it plainly: a certificate-using system “MUST reject the certificate if it encounters a critical extension it does not recognize or a critical extension that contains information that it cannot process. A non-critical extension MAY be ignored if it is not recognized, but MUST be processed if it is recognized.”

Read that twice, because both halves catch people out. Marking an extension critical is not a way of emphasising it; it is an instruction to fail closed on any verifier that has never heard of it, which is precisely what happens when a modern issuing profile meets an embedded client from 2014. Marking an extension non-critical does not make it advisory either. A verifier that understands the extension is obliged to honour it, so a non-critical extendedKeyUsage still blocks the wrong purpose on every mainstream stack.

The same section adds a rule worth remembering during any templating exercise: “A certificate MUST NOT include more than one instance of a particular extension.” Two subjectAltName extensions is not a richer certificate, it is a malformed one.

basicConstraints draws the line between a CA and everything else

basicConstraints carries a boolean and an optional integer.

BasicConstraints ::= SEQUENCE {
     cA                BOOLEAN DEFAULT FALSE,
     pathLenConstraint INTEGER (0..MAX) OPTIONAL }

The cA boolean is the authority switch. When it is absent or false, RFC 5280 §4.2.1.9 states that “the certified public key MUST NOT be used to verify certificate signatures”, and the keyCertSign bit “MUST NOT be asserted”. This is the check that stops an ordinary server certificate from being used to mint further certificates, and it is why a leaf certificate accidentally promoted into a trust store still cannot issue anything.

pathLenConstraint is the field most often miscounted. It “gives the maximum number of non-self-issued intermediate certificates that may follow this certificate in a valid certification path”, and the leaf is explicitly excluded from that count. So pathlen:0 on an intermediate does not mean the intermediate is useless; it means the intermediate may issue end-entity certificates and nothing else. A root built with pathlen:1 permits exactly one intermediate beneath it before the leaf.

openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:4096 -out root.key
openssl req -x509 -new -key root.key -sha256 -days 3650 \
  -subj "/O=RunBook Academy Lab/CN=RunBook Lab Root CA" \
  -addext "basicConstraints=critical,CA:TRUE,pathlen:1" \
  -addext "keyUsage=critical,keyCertSign,cRLSign" -out root.crt

Note that the two constraints are declared together. That is not stylistic: pathLenConstraint is meaningful only when cA is asserted and keyCertSign is present, and RFC 5280 forbids a CA from including it otherwise. A root carrying a path length but no keyCertSign bit is a configuration mistake that some verifiers will reject and others will silently ignore, which is the worst possible combination.

keyUsage and extendedKeyUsage answer different questions

These two are routinely conflated because both look like permission lists. They are not the same kind of permission.

• keyUsage is about the key. It is a bit string naming the cryptographic operations the public key may take part in: digitalSignature, keyEncipherment, keyAgreement, keyCertSign, cRLSign and others. It says nothing about protocols.
• extendedKeyUsage is about the role. It is a list of object identifiers naming application purposes, such as TLS Web Server Authentication or TLS Web Client Authentication. It says nothing about algorithms.

The key type constrains which keyUsage bits are even coherent. Under the CA/Browser Forum TLS Baseline Requirements, an ECDSA subscriber certificate MUST assert digitalSignature and is not permitted to assert keyEncipherment at all, because an ECDSA key cannot encipher a key in the first place. For RSA the requirements are looser but not liberal: at least one usage must be set, keyAgreement, keyCertSign and cRLSign are not permitted in a subscriber certificate, and dataEncipherment is marked not recommended and flagged as a pending prohibition. Copying an RSA extension block onto an ECDSA profile is one of the most common ways to produce a certificate that a strict verifier rejects and a lax one accepts.

On a CA certificate the same extension carries the two bits that make an authority an authority: keyCertSign, which permits the key to sign certificates, and cRLSign, which permits it to sign revocation lists. Those are separate permissions, and a hierarchy that asserts only the first has an authority that cannot legitimately revoke anything it issues. The consequences of that omission recently became sharper. RFC 10007, published in June 2026, updates RFC 5280 because the original revocation-list validation algorithm “does not explicitly include a corresponding check for the presence of the keyUsage certificate extension”, and requires verifiers to check for cRLSign when validating a CRL. A CA certificate missing that bit will therefore have its revocation lists rejected by conforming verifiers, and the symptom will present as revocation checking silently doing nothing rather than as an error anyone notices.

On the extendedKeyUsage side, the Baseline Requirements make id-kp-serverAuth mandatory for a TLS server certificate and id-kp-clientAuth optional. The catch-all anyExtendedKeyUsage identifier MUST NOT appear, and neither may codeSigning, emailProtection, timeStamping or OCSPSigning. Reaching for the catch-all to make a mutual-TLS problem disappear produces a certificate that public CAs will refuse to issue and that some verifiers treat as having no permitted purpose at all.

The failure when the purpose is wrong

Extension faults are best learned by producing them. openssl verify accepts a -purpose argument that asks the same question a TLS stack asks, and it names the answer precisely.

openssl verify -CAfile root.crt -untrusted srv-ca.crt -purpose sslserver app.crt
openssl verify -CAfile root.crt -untrusted srv-ca.crt -purpose sslclient app.crt

The certificate under test carries TLS Web Server Authentication and nothing else, so the two runs diverge:

app.crt: OK
error 26 at 0 depth lookup: unsuitable certificate purpose

The first line is the server-purpose run, exiting zero. The second is the client-purpose run, exiting 2. Two details in that error repay attention. at 0 depth says the fault is in the leaf itself rather than anywhere up the chain, which immediately rules out the intermediate and the root. And the wording is about suitability, not validity: the certificate is perfectly well formed, correctly signed and inside its validity window. It is simply not the kind of certificate being asked for. That distinction is the difference between reissuing with a corrected profile and hunting a chain problem that does not exist.

flowchart TD
    A["Verifier reads an extension"] --> B{"Recognised?"}
    B -- "no, and critical" --> R["Reject the certificate"]
    B -- "no, and non-critical" --> S["Skip it"]
    B -- "yes" --> P["Process it and enforce it"]
    P --> C{"Purpose requested is listed?"}
    C -- "yes" --> OK["Continue path validation"]
    C -- "no" --> U["error 26: unsuitable certificate purpose"]

The diagram compresses the whole decision. Recognition and criticality decide whether an extension is enforced or skipped, and only after that does the content of the extension get compared against what the application is asking for. An operator who remembers those two stages can predict, from the certificate alone, which clients will accept it and which will not.

Production discipline

1. Template extensions per key type, not per service. Keep separate ECDSA and RSA extension files so that keyEncipherment cannot leak onto an elliptic-curve profile during a copy.
2. Check the purpose before you check the chain. One openssl verify -purpose run distinguishes a profile fault from a trust fault, and it takes a second.
3. Never add a leaf to a trust store to clear an error. It hides a missing intermediate, and it grants the leaf holder authority you did not intend to delegate.
4. Set pathlen deliberately and document the count. Write down that it counts intermediates below the certificate and excludes the leaf, because the next person to extend the hierarchy will be working from your note rather than from the RFC.

Cross-course references

• Kubernetes for Production Sysadmins - Part LXXVI (Certs) covers cluster components that present the same certificate for both server and client roles, which is only legal when both purposes are listed.
• Linux for Production Sysadmins - Part LXXI (TLS) covers the host view of authorities and chains, where a CA certificate missing keyCertSign or cRLSign stops behaving as an authority.
• Observability for Production Sysadmins - Part XVIII (AlertingRules) covers rule anatomy and hysteresis, which turns a handshake failure from a bad profile into a page rather than a ticket.

Quiz