Skip to main content
RunBook Academy

← All break/fix scenarios in OPNsense

advancedHigh availability~30 min

Break/Fix: Both HA Nodes Claim MASTER

Reported symptoms

  • ●Both nodes report carp: MASTER for the same VHID on the interface the VIP is bound to
  • ●Duplicate IP / MAC entries appear in downstream switches' MAC tables
  • ●Intermittent packet loss as switches flap between the two nodes

Evidence

  • · ifconfig igb0 on both nodes shows carp: MASTER for VHID 1, and net.inet.carp.demotion is 0 on both
  • · tcpdump -n -i igb0 proto carp on each node shows only that node's own advertisements for the VHID
  • · The pfsync link is healthy — ifconfig pfsync0 reports syncok: 1 and the two state counts match
Diagnosis and resolutionclick to reveal

Root cause

Each node is not receiving the other's CARP advertisements on the segment the VIP is bound to, so each concludes it is alone and promotes itself to MASTER. CARP advertisements travel on the parent interface of each VIP — the LAN, WAN or DMZ interface — and never on the pfsync sync link, so a healthy sync link neither causes this fault nor rules it out. The break is on the VIP parent segment (switch port, VLAN, port isolation, multicast filtering) or in the CARP password, which makes each node reject what the other sends.

Remediation

1. On each node, run ifconfig against the VIP parent interface and confirm both report carp: MASTER for the same VHID. 2. Run tcpdump -n -i <parent> proto carp on both nodes, on that same parent interface. If each node sees only its own advertisements, the parent segment is broken — check the switch port, the VLAN, port isolation and multicast filtering for 224.0.0.18. If each node sees the other's advertisements and both stay MASTER, check netstat -s -p carp for a rising "discarded for bad authentication" count and compare the CARP password on both nodes. 3. Repeat for every VIP parent interface; a pair can be split on one segment and healthy on another. 4. Once the cause is known, demote one node — Interfaces → Virtual IPs → Status → Enter Persistent CARP Maintenance Mode — so the network settles on a single master while the segment is repaired.

Verification

Only one node reports carp: MASTER for each VHID and the other reports BACKUP; a capture on each VIP parent interface shows advertisements from both nodes; switches show a single MAC for each VIP; downstream traffic uses one path. Run a controlled failover test.

Prevention

Make sure every interface that carries a CARP VIP passes protocol CARP and is not filtering the multicast group 224.0.0.18 — that is the path the election actually runs on. Separately, use a dedicated HA sync interface with its own VLAN and firewall rules that pass IP protocol pfsync (240) between the two sync addresses; pfsync has no port, so there is no port to open. Monitor CARP state changes per parent interface and alert when one VHID shows two MASTERs. Add "node isolation" checks to your change window so a misconfigured switch port cannot silently break the election.

Scenario

You are operating an OPNsense HA pair in active/passive. The following symptoms appear:

  • Both nodes report carp: MASTER for the same VHID on the interface the VIP is bound to
  • Duplicate IP / MAC entries appear in downstream switches’ MAC tables
  • Intermittent packet loss as switches flap between the two nodes

Available evidence:

  • ifconfig igb0 on both nodes shows carp: MASTER vhid 1, and net.inet.carp.demotion is 0 on both
  • tcpdump -n -i igb0 proto carp on each node shows only that node’s own advertisements for the VHID
  • The pfsync link is healthy — ifconfig pfsync0 reports syncok: 1 and the two state counts match

Your task

Determine the cause, recover, document, and validate.

Investigation

The investigation follows the discipline taught in Part XXVII:

  1. Form hypothesis, find evidence, test, validate.
  2. Use the available evidence above to bound the search.
  3. Reach one of the likely root causes.

Recovery procedure

(Do not reveal until you have reasoned through the problem.)

  1. Identify the failing component.
  2. Apply the remediation pathway.
  3. Validate with the verification step.
  4. Document the incident.

Remediation

  1. On each node, run ifconfig against the VIP parent interface and confirm both report carp: MASTER for the same VHID. 2. Run tcpdump -n -i <parent> proto carp on both nodes, on that same parent interface. If each node sees only its own advertisements, the parent segment is broken — check the switch port, the VLAN, port isolation and multicast filtering for 224.0.0.18. If each node sees the other’s advertisements and both stay MASTER, check netstat -s -p carp for a rising “discarded for bad authentication” count and compare the CARP password on both nodes. 3. Repeat for every VIP parent interface; a pair can be split on one segment and healthy on another. 4. Once the cause is known, demote one node — Interfaces → Virtual IPs → Status → Enter Persistent CARP Maintenance Mode — so the network settles on a single master while the segment is repaired.

Verification

Only one node reports carp: MASTER for each VHID and the other reports BACKUP; a capture on each VIP parent interface shows advertisements from both nodes; switches show a single MAC for each VIP; downstream traffic uses one path. Run a controlled failover test.

Rollback

Clear the demotion on the node that was demoted — press Enter Persistent CARP Maintenance Mode a second time, which subtracts the 240 it added — and confirm both nodes return to their intended roles.

Prevention

Make sure every interface that carries a CARP VIP passes protocol CARP and is not filtering the multicast group 224.0.0.18 — that is the path the election actually runs on. Separately, use a dedicated HA sync interface with its own VLAN and firewall rules that pass IP protocol pfsync (240) between the two sync addresses; pfsync has no port, so there is no port to open. Monitor CARP state changes per parent interface and alert when one VHID shows two MASTERs. Add “node isolation” checks to your change window so a misconfigured switch port cannot silently break the election.