Skip to main content
RunBook Academy

← All runbooks in OPNsense

high riskcluster affecting~120 min

Runbook: Upgrade an HA Pair

1 · Prerequisites

Confirm every item is in place before any state change.

  • Two-node OPNsense HA pair in steady state
  • Maintenance window scheduled with the operations team
  • Backup of the running configuration on both appliances
  • Release notes for the target version reviewed

2 · Pre-checks

Read-only diagnostic commands. If any of these don't match expected output, stop and investigate further.

  • · Confirm both appliances are in the expected master/backup roles and the cluster is healthy
  • · Capture a config backup of both appliances under System > Configuration > Backups
  • · Review the release notes for the target version and confirm there are no breaking changes that affect the running configuration
  • · Confirm the second appliance is on the same firmware as the primary, since misaligned firmwares can break config sync

3 · Procedure

Execute each step in order. Verify the expected output of a step before moving to the next.

  1. 1Disable config sync on the primary so the upgrade does not push a new config to the still-old secondary (re-enable after the upgrade completes)
  2. 2Perform the HA maintenance procedure to move the CARP VIPs to the secondary so the primary is the backup
  3. 3On the primary appliance, run the firmware upgrade from System > Firmware > Updates and reboot when prompted
  4. 4After the primary returns, confirm it is the backup and the cluster is healthy with the new firmware on the primary
  5. 5Observe the cluster for at least 30 minutes to confirm no regressions or alerts are firing
  6. 6Re-enable config sync on the primary and confirm the configuration is replicated to the secondary
  7. 7Move the VIPs back to the primary so the primary is the master again
  8. 8Disable config sync on the secondary and perform the firmware upgrade on the secondary
  9. 9After the secondary returns, confirm it is the backup and the cluster is healthy with the new firmware on the secondary
  10. 10Re-enable config sync on both appliances and confirm the configuration is fully replicated
  11. 11Force a manual failover and confirm both appliances can take over the master role cleanly

4 · Verification

Confirm the procedure actually fixed the problem.

  • Both appliances are on the new firmware version and the cluster is in the expected master/backup roles
  • Config sync replicates a test change from the primary to the secondary
  • A forced failover on the primary causes the secondary to assume the VIPs within a few seconds
  • No release-note-driven regressions are observed in the dashboard or the firewall log

5 · Rollback

If verification fails, undo the procedure in reverse order.

  • If the primary fails to return from the upgrade, leave it offline and let the secondary continue as master
  • If the secondary fails to return from the upgrade, restore the previous firmware image from System > Firmware > Updates
  • If the upgrade introduces a configuration problem, restore the previous config backup

6 · Escalation

When the runbook isn't enough, contact:

  • · Engage the platform team if the firmware upgrade fails on the appliance and the appliance does not boot
  • · Engage the network team if the cluster enters a split-brain state after the upgrade
  • · Engage the on-call engineer if a release-note-driven regression causes a sustained outage

Purpose

Upgrade both appliances in an OPNsense HA pair to a new firmware version without a sustained outage, by upgrading one node at a time and validating the cluster between the two upgrades.

When to use this runbook

Use this runbook when the firmware version is out of date, when a security advisory requires an upgrade, or when a new feature requires the new firmware on both appliances. Always review the release notes before scheduling the upgrade.

Pre-checks

Before starting the procedure, confirm the prerequisites and pre-checks are met. The structured lists are rendered from the frontmatter by the page layout.

Procedure

Follow the steps from the frontmatter procedure steps. The page layout renders the steps as a checklist with copy-to-clipboard affordances.

Verification

After the procedure, the structured verification items from the frontmatter are rendered as a checklist.

Rollback

If the procedure fails or makes things worse, follow the structured rollback steps from the frontmatter.

Escalation

The structured escalation path is rendered from the frontmatter. Use it if the operator cannot complete the procedure safely.

References

  1. OPNsense documentation — firmware upgrades
  2. OPNsense documentation — High Availability
  3. OPNsense blog — release announcements