Skip to main content
RunBook Academy

Secrets, PKI & CertificatesVIII · TLS TroubleshootingTroubleshooting

Why disabling verification is not a fix

Advanced⏱ ~24 mincurlopensslopenssh

What you'll learn

  • State exactly which checks each verification escape hatch removes
  • Explain why an encrypted session to an unauthenticated peer carries no guarantee
  • Match every failure class to the repair that keeps verification switched on
  • Prevent a temporary exception from becoming a permanent estate-wide default

Prerequisites

Verified against OpenSSL 3.5.x teaching target; 3.0+ minimum · OpenSSH 10.x teaching target; 8.2+ minimum for certificate workflows · OpenBao 2.6.x · Smallstep step-ca 0.30.x · Certbot / Pebble Certbot current release; Pebble 2.10.x ACME test server · Kubernetes (cross-course target) 1.36.x · PostgreSQL 17.x · 2026-08-26

Not yet marked complete on this device.

Every engineer knows the three keystrokes that make a certificate error stop. They are reached for at the worst possible moment, by tired people under pressure, and they work immediately, which is precisely the problem. This lesson is about what they actually switch off, and about the repair that was available in each case and took about the same amount of typing.

Encryption is not authentication

A TLS session provides two separable properties. Confidentiality comes from session keys derived from an ephemeral key exchange, and it is unaffected by any of the options discussed here. Authentication comes from the certificate: it is the only thing that binds an identity you asked for to the key that the peer proves it holds.

Turn verification off and the first property remains while the second disappears. The traffic is still encrypted, the tooling still reports a successful handshake, and the client has no idea who is on the other end. An interceptor with any key at all now satisfies every remaining requirement.

flowchart LR
    A["client with verification off"] --> B["peer presents any certificate"]
    B --> C["handshake completes, traffic is encrypted"]
    C --> D["the identity of the peer is never established"]
    D --> E["an interceptor reads and rewrites the plaintext"]
    D --> F["the client reports success in either case"]

The diagram has no failure branch, and that is its whole point. A client that does not verify cannot distinguish the correct service from an impostor, so both paths lead to a green result. Whatever confidence the deployment placed in TLS has been removed while every visible indicator that it is present remains in place.

curl -k and what the single character removes

The option disables peer certificate verification and the hostname check together. curl proceeds no matter what the verification produced, which means it also stops reporting exit status 60, so every script that branches on the exit status loses its ability to detect the fault at all.

# ANTI-PATTERN. Do not copy. Disables peer and hostname
# verification, so any key from any peer is accepted.
curl -k https://api.example.com/health

# ANTI-PATTERN. Do not copy. Accepts a changed host key and
# degrades the session as described further down this page.
ssh -o StrictHostKeyChecking=no deploy@web-01

Engineers reach for it for understandable reasons. It removes the error in one character, it appears in the top answer to almost every search, and during an incident it seems to prove something useful about whether the service is up. The last of those is the most damaging belief, because a health check that cannot fail on a certificate fault will report green through an expired certificate, a wrong certificate and an active interception with equal enthusiasm.

The lasting cost is that the option is sticky. It enters a health check or a pipeline step during one incident and stays for years, and by the time anyone notices, an entire internal CA has expired without a single alert firing.

verify=False, and the warning that gets suppressed next

In application code the same switch is a keyword argument. It disables anchor validation and the hostname check for every request made through that session, and the underlying HTTP library raises an insecure-request warning to say so. What happens next is the part worth naming: the warning is noisy, so somebody suppresses the warning, and the last remaining signal that the deployment is unauthenticated disappears into a configuration file.

The correct form is not more work. The same argument accepts a path to the anchor bundle that this service is supposed to trust, which keeps verification on and additionally documents the trust decision in the code where a reviewer can see it.

# ANTI-PATTERN: verify=False disables anchor validation and the
# hostname check for every request through this session.
# requests.get("https://api.example.com/health", verify=False)

# Correct: nominate the anchor set this service must trust.
import requests

requests.get(
    "https://api.example.com/health",
    verify="/etc/ssl/certs/internal-root.pem",
)

Every runtime has an equivalent, and some offer a single environment variable that disables verification for an entire process. Those are worse than the per-request form, because they apply to calls the author never considered, including calls made by libraries deep inside the dependency tree.

StrictHostKeyChecking=no is not a prompt setting

This one is routinely described as a way to skip an interactive question, and that description is wrong in a way that matters. On genuine first contact it does accept an unknown key without asking. On a changed key it does something far more serious: it accepts the new key and then silently disables password authentication, keyboard-interactive authentication, agent forwarding, X11 forwarding, port and tunnel forwarding, and the mechanism that would otherwise learn additional host keys.

Consider what that means during an incident. A changed host key is exactly the event an interception produces. The setting accepts it, and then produces a cluster of secondary failures that look nothing like a trust problem, so the investigation goes hunting for a forwarding bug while the actual finding sits unexamined.

For automation that genuinely meets a host for the first time, the honest option is accept-new, which trusts an unknown key and still refuses a changed one. Better still is to remove the first-contact problem entirely by publishing host keys, or by trusting a host certificate authority so that a rebuilt host presents a certificate rather than a surprise.

# Genuine first contact in automation: accept a new key, still
# refuse a changed one.
ssh -o StrictHostKeyChecking=accept-new deploy@web-01

# Better: know the host key before the first connection.
ssh-keygen -F web-01

One footnote saves time later. The familiar suggestion to run a key-removal command, printed alongside the changed-key warning, is added by distribution patches. Upstream prints only the file and line of the offending key and a line telling you to add the correct host key. If you are reading a warning that lacks the friendly hint, nothing is wrong with your build.

The correct fix for each class

Failure classWhat disabling verification hidesThe repair
Missing intermediateA server under-sending its chain to every clientServe the full chain file, then re-count what arrives
Unknown anchorA trust decision nobody has made deliberatelyInstall the anchor, or correct which CA issued the leaf
Wrong hostnameA certificate belonging to a different serviceReissue with the name in the SAN, or request a name it carries
ExpiredAn expiry that will recur on the same day next cycleConfirm the clock, then renew and reload
Unsuitable purposeAn issuance profile with the wrong usageReissue with the extended key usage the role needs
Changed SSH host keyA rebuild, or an interception, indistinguishablyVerify the fingerprint out of band, then update trust

The anchor case is the one people believe is hard, and it is two commands on a Debian-family host.

# Install a private trust anchor so verification can succeed.
sudo cp root.crt /usr/local/share/ca-certificates/runbook-lab-root.crt
sudo update-ca-certificates

# Then verify against that anchor set instead of switching off.
curl -sS --cacert /etc/ssl/certs/ca-certificates.crt \
  -o /dev/null https://api.example.com/health
printf 'verification result: %s\n' "$?"

Notice what the second command preserves. The request still fails if the certificate is wrong, and it still fails if somebody replaces the endpoint, which is the entire reason the check exists.

Production discipline

  1. Treat every escape hatch as a security incident, not a workaround. If one is used, it needs a ticket, a named owner and a removal date recorded at the moment it is typed.
  2. Never let one into a health check or a probe. A check that cannot fail on a certificate fault has been converted into a check that lies.
  3. Scope any exception to one host and one call. A process-wide environment variable applies to dependencies the author never reviewed.
  4. Search for them in review and in CI. The strings are short, distinctive and easy to reject automatically; a repository that has never been searched almost always has several.
  5. Prefer nominating an anchor set to removing the check. It is the same amount of typing, it keeps authentication intact, and it records the trust decision where a reviewer can read it.

Cross-course references

  • Linux for Production Sysadmins - Part XXVI (SSH) covers host key distribution and the known-hosts mechanisms that remove the temptation to accept whatever key arrives.
  • Git, CI/CD & GitOps for Infrastructure Engineers - Part CIII (InfraRepoAnti) covers catching this class of shortcut in review, where a one-character flag is easy to miss and cheap to reject.
  • Observability for Production Sysadmins - Part CXI (AntiPatterns) covers monitoring that cannot fail, of which a probe with verification disabled is the canonical example.

Quiz

Knowledge check · 4 questions

  1. Q1. A health check runs with certificate verification disabled. What has that check lost the ability to detect?

  2. Q2. On a changed host key, StrictHostKeyChecking set to no accepts the new key and also silently disables password authentication, agent forwarding and port forwarding.

  3. Q3. Why does a handshake still complete successfully when certificate verification has been switched off?

  4. Q4. Replace an insecure workaround with a repair that keeps verification switched on.

    A deployment pipeline calls an internal API at internal.example.com with certificate verification disabled. The comment beside it says added during the October incident. A capture shows the server transmits a leaf and an intermediate, the leaf carries internal.example.com in its SAN, and the chain terminates at a private root that is not present in the pipeline runner image.

Passing score: 75%. Answers are checked in this browser.