Certificate expiry monitoring — the alert that prevents the outage

Certificates expire. A certificate that expired an hour ago is not a certificate — it is a hard outage. Browsers refuse the connection, TLS clients error out, services that depend on TLS (every API, every internal web app, every IPsec peer with certificate auth, every syslog target with TLS) stop working.

Renewal automation reduces the chance of expiry but does not eliminate it. ACME can fail because the CA is down, the DNS provider API token is revoked, the firewall rule blocks the challenge response, or the certificate was never set up to auto-renew in the first place. Expiry monitoring is the safety net that catches the failures automation missed.

This lesson covers why expiry monitoring is non-optional, how to monitor expiry from inside OPNsense and from external probes, how to set thresholds and escalation, and how to diagnose silent renewal failures using expiry data.

Why certificates expire

The validity field in an X.509 certificate is bounded by notBefore and notAfter. The certificate is valid only inside that window. Outside the window, the certificate is not valid — modern TLS clients refuse to connect.

Why bound the validity at all? Two reasons:

1. Limit the blast radius of key compromise. A certificate valid for 10 years means a stolen private key is useful for 10 years. A certificate valid for 90 days means the stolen key is useful for at most 90 days.
2. Force operational discipline. If certificates must be renewed periodically, the infrastructure that issues, distributes, and installs them must be working. A certificate that never expires can hide a broken renewal pipeline for years.

The trade-off: short-lived certificates are safer but require the renewal pipeline to work. Expiry monitoring catches the pipeline failures.

$ openssl s_client -connect fw.example.com:443 -servername fw.example.com </dev/null 2>/dev/null | openssl x509 -noout -dates -checkend 2592000

notBefore=Aug  1 00:00:00 2026 GMT
notAfter=Oct 30 00:00:00 2026 GMT
Certificate will not expire within 2592000 seconds (30 days)

Monitoring from inside OPNsense

OPNsense includes Monit under Services → Monit. Monit can run checks against services and alert when thresholds are crossed. For certificate expiry, a Monit check can be defined that runs an openssl command against the local certificate store and alerts when the remaining validity is below a threshold.

The configuration:

1. Open Services → Monit → Tests.
2. Add a new test:
   • Name: e.g. GUI certificate expiry.
   • Condition: failed content = "will not expire" — this matches when openssl x509 -checkend reports the certificate will not expire within the configured window.
   • Action: alert via the configured Monit alert target (email, webhook, etc.).
3. Link the test to a service or run it on a schedule.

A simpler approach that does not require Monit-specific syntax: a custom script in /usr/local/sbin/ that runs on the cron schedule, checks each certificate’s expiry, and emails or logs when one is below threshold. Cron can be configured under System → Settings → Cron.

# cat > /usr/local/sbin/check-cert-expiry.sh <<'EOF'
#!/bin/sh
THRESHOLD=30
for cert in /var/etc/cert-manager/*.crt; do
name=$(basename "$cert" .crt)
expiry=$(openssl x509 -in "$cert" -noout -enddate | cut -d= -f2)
echo "$name expires: $expiry"
openssl x509 -in "$cert" -noout -checkend $((THRESHOLD*86400)) >/dev/null 2>&1 ||   logger -p auth.err "CERT EXPIRY WARNING: $name expires $expiry"
done
EOF
chmod +x /usr/local/sbin/check-cert-expiry.sh
/usr/local/sbin/check-cert-expiry.sh

fw.example.com expires: Oct 30 00:00:00 2026 GMT
vpn.example.com expires: Aug 12 00:00:00 2026 GMT
CERT EXPIRY WARNING: vpn.example.com expires Aug 12 00:00:00 2026 GMT

Monitoring from outside OPNsense

Internal monitoring catches expiry of certificates OPNsense knows about. It misses certificates on systems outside OPNsense’s view: remote endpoints, third-party APIs, services hosted elsewhere in the estate. External monitoring covers these.

The pattern: an external probe connects to the service over TLS, retrieves the certificate, and reports the remaining validity. Tools:

• Prometheus blackbox_exporter with the tls probe — the probe_ssl_earliest_cert_expiry metric is the time the certificate expires.
• check_ssl_cert (a Nagios plugin) — checks expiry and reports via Nagios.
• Custom scripts with openssl s_client and x509 -checkend — the same one-liner as inside OPNsense, run from a probe host.
• SSL monitoring services — SSL Labs, statuspage vendors, and dedicated cert monitoring services.

For an estate of any size, Prometheus + blackbox_exporter + Alertmanager is the production answer. The probe runs from outside the firewall (so it validates what a real client sees), exports a metric, and Alertmanager fires when the remaining validity crosses a threshold.

Thresholds and escalation

A single threshold (“alert when expiry < 30 days”) is a starting point. Production escalation uses multiple thresholds:

The escalation is more important than any single threshold. A 60-day warning gives the operator a week to fix a broken renewal pipeline before it becomes urgent. A 7-day page is the last line of defence.

Diagnosing renewal failures

When the monitoring fires, the diagnostic path:

1. Confirm the certificate’s current state. Use openssl s_client against the live service. Confirm what is actually being presented.
2. Check the renewal automation logs. Under Services → ACME Client → Log or in the system log. Look for the most recent run and any errors.
3. Run the renewal manually. Trigger the automation from the GUI or by running the ACME client on the CLI. See whether the failure reproduces.
4. Check the DNS API token. For DNS-01 challenges, the token’s validity and scope. A revoked or scoped-down token produces a clean-looking error from the DNS provider.
5. Check the firewall rules. For HTTP-01, the rule that allows inbound port 80 to the firewall from the CA. A recent rule change can block the challenge.
6. Re-issue manually. If the renewal pipeline is broken, issue a certificate manually (via the OPNsense GUI or openssl), install it, and have expiry monitoring cover the gap until the renewal pipeline is fixed.

Summary

• Certificates expire. An expired certificate is an outage, not a degraded service.
• Monitor expiry from inside OPNsense (Monit, cron scripts) for certificates OPNsense knows about, and from external probes for end-to-end validity.
• Use multiple thresholds (60, 30, 14, 7 days) with escalating severity. The renewal pipeline should have re-issued long before the page fires.
• Diagnose renewal failures by checking the automation logs, running the renewal manually, and validating the DNS API token and firewall rules.