PBR anti-patterns — PBR everywhere, conflicting route-maps, missing table, asymmetric PBR

PBR is a powerful tool. Like every powerful tool, it has a set of production anti-patterns that emerge when the operator uses it without sufficient restraint. The anti-patterns are not bugs in FRR or VyOS — they are design choices that look reasonable at configuration time but create operational trouble during incidents, during scaling, and during handoff to the next operator. This lesson walks through the most common PBR anti-patterns, the operational signal that flags each one, and the remediation that the engineer must apply.

Anti-pattern 1 — PBR everywhere

The operator reaches for PBR to solve every routing problem. Source-based routing for voice, source-based routing for data, source-based routing for management, source-based routing for guests, source-based routing for IoT. The result: every flow on the router is PBR-routed; the main table is consulted only for host-local destinations.

flowchart TB
  subgraph ALL["PBR-everywhere"]
    R1[Voice: PBR via provider-A]
    R2[Data: PBR via provider-B]
    R3[MGMT: PBR via MGMT uplink]
    R4[Guest: PBR via guest VLAN]
    R5[IoT: PBR via IoT VLAN]
    R6[Default: main table]
  end
  R1 --> NET1[Provider-A]
  R2 --> NET2[Provider-B]
  R3 --> NET3[MGMT uplink]
  R4 --> NET4[Guest VLAN]
  R5 --> NET5[IoT VLAN]
  R6 --> NET6[Default route]

The operational smell:

• The router has more ip rule entries than the kernel rule selector can walk in O(1). The data plane degrades.
• Every change to a route-map must be coordinated with every team that owns a traffic class. Change windows become political.
• The post-incident review cannot determine the effective forwarding path without reading every route-map in order.
• New operators cannot understand the routing without a runbook.

The remediation:

• Use PBR only where destination-based routing cannot express the policy. For source-based provider selection, prefer VRFs + per-VRF uplinks. For application routing, prefer DSCP + QoS. For compliance tagging, prefer BGP community propagation.
• When PBR is necessary, use the minimum number of route-maps. Combine related source prefixes in one prefix-list and one route-map.

Anti-pattern 2 — conflicting route-maps

Two route-maps bound to the same interface, each setting a different next-hop for overlapping source prefixes. The rule priority determines which wins; the operator did not intend one to win.

configure
set policy route-map RM-VOICE-OUT rule 10 action permit
set policy route-map RM-VOICE-OUT rule 10 match ip source address prefix-list PL-VOICE-SRC
set policy route-map RM-VOICE-OUT rule 10 set ip next-hop '198.51.100.1'
set policy route-map RM-OVERRIDE rule 10 action permit
set policy route-map RM-OVERRIDE rule 10 match ip source address prefix-list PL-OVERRIDE
set policy route-map RM-OVERRIDE rule 10 set ip next-hop '203.0.113.1'
set interfaces ethernet eth1 policy route-map RM-VOICE-OUT
set interfaces ethernet eth1 policy route-map RM-OVERRIDE
commit
save

The two route-maps are bound to the same interface. FRR generates two ip rule entries, one for each route-map. The rule priority depends on the route-map’s hash; the operator cannot easily predict which wins.

The operational smell:

• The voice traffic sometimes goes out provider-A and sometimes out provider-B. The behaviour is non-deterministic.
• ip rule show shows two PBR rules at similar priorities; the operator cannot tell which one wins for a specific source.
• A change to either route-map changes the behaviour of the other.

The remediation:

• Combine the two route-maps into one. Add a new rule for the override prefix; the route-map evaluation order determines which set clause fires.
• Or, use a single route-map and bind it to a single interface.

configure
delete policy route-map RM-OVERRIDE
set policy route-map RM-VOICE-OUT rule 15 action permit
set policy route-map RM-VOICE-OUT rule 15 match ip source address prefix-list PL-OVERRIDE
set policy route-map RM-VOICE-OUT rule 15 set ip next-hop '203.0.113.1'
commit
save

The combined route-map has rule 10 for voice, rule 15 for override. The sequence number determines which rule fires; the operator has explicit control.

Anti-pattern 3 — missing PBR table

The route-map has a set ip next-hop clause, but the operator forgot to bind the route-map to an interface. The configuration is technically valid; the route-map is defined; no rule is installed.

configure
set policy route-map RM-VOICE-OUT rule 10 action permit
set policy route-map RM-VOICE-OUT rule 10 match ip source address prefix-list PL-VOICE-SRC
set policy route-map RM-VOICE-OUT rule 10 set ip next-hop '198.51.100.1'
# NOTE: no `set interfaces ethernet eth1 policy route-map RM-VOICE-OUT` line
commit
save

The route-map exists in the configuration tree. FRR has the route-map definition. But no ip rule is installed because no interface binding exists. The PBR is silently inactive.

The operational smell:

• show policy route-map shows the route-map; the operator believes it is active.
• ip rule show has no PBR rule.
• Voice traffic takes the main-table path; the operator wonders why.

The remediation is the missing interface binding. Add the line:

set interfaces ethernet eth1 policy route-map RM-VOICE-OUT
commit
save

After commit, FRR installs the ip rule and populates the PBR table.

A more pernicious variant: the route-map is bound to an interface but the rule has only action deny and no action permit with a set ip next-hop. The PBR table is empty; the rule consults an empty table and falls through.

set policy route-map RM-VOICE-OUT rule 10 action deny
set policy route-map RM-VOICE-OUT rule 10 match ip source address prefix-list PL-VOICE-SRC

The route-map denies all matching traffic. No set ip next-hop clause exists. The rule is installed; the table is empty; the matching traffic is dropped.

The remediation: add an action permit rule with a set ip next-hop clause.

Anti-pattern 4 — asymmetric PBR breaking stateful firewalls

The forward path uses PBR (e.g. voice traffic via provider-A). The return path uses the main table (response via provider-B). The stateful firewall on the return path drops the packets because no state exists.

sequenceDiagram
  participant Src as Voice source
  participant FW1 as Forward firewall
  participant V as Voice provider
  participant Dst as Destination
  participant FW2 as Return firewall
  Src->>FW1: SYN (PBR: via voice provider)
  FW1->>V: Forward via eth2
  V->>Dst: SYN forwarded
  Dst-->>FW2: SYN-ACK (return path)
  FW2-->>Src: SYN-ACK dropped (no state)
  Note over FW2: Asymmetric path: forward on eth2,<br/>return on eth0

The operational smell:

• Voice calls have one-way audio or hang after the initial signalling.
• tcpdump on the egress interface of the forward path shows the forward packets; tcpdump on the egress interface of the return path shows the return packets.
• The firewall logs show dropped SYN-ACK packets.

The remediation:

• Apply the same PBR rule to the return path. Bind the route-map to the interface that receives the return traffic.
• If the return traffic enters on the same interface as the forward traffic, the rule is already covering both directions (the iif eth1 match is symmetric for packets received on eth1).
• If the return traffic enters on a different interface, configure a separate route-map for that interface that matches the response and sets the next-hop to the same provider.
• Alternatively, disable stateful firewall on the affected interfaces if stateful inspection is not required.

Anti-pattern 5 — route-maps bound to too many interfaces

The operator binds the same route-map to every interface on the router. The PBR rule fires for traffic on every interface, including management, loopback, and provider-facing interfaces.

set interfaces ethernet eth0 policy route-map RM-VOICE-OUT
set interfaces ethernet eth1 policy route-map RM-VOICE-OUT
set interfaces ethernet eth2 policy route-map RM-VOICE-OUT
set interfaces ethernet eth3 policy route-map RM-VOICE-OUT

The operational smell:

• The PBR rule fires for traffic that should not be subject to the policy (e.g. management traffic destined for the router’s own loopback).
• ip rule show has a long list of PBR rules, each with iif matching a different interface.
• The router’s own traffic (e.g. SNMP polls, BGP sessions) is PBR-routed; the management traffic takes an unexpected path.

The remediation:

• Bind the route-map only to the interfaces that carry the traffic class the policy is intended for. Management interfaces should not have a data-traffic route-map bound.
• Use the match interface clause within the route-map to restrict the rule to traffic that arrived on a specific interface, rather than relying on the interface binding to do the filtering.

Anti-pattern 6 — hard-coded next-hop that fails

The PBR rule hard-codes a next-hop (set ip next-hop X.X.X.X). When the next-hop fails (the upstream router is down, the link is down), the PBR rule does not fail over. The kernel still forwards to the dead next-hop; the packets disappear.

set policy route-map RM-VOICE-OUT rule 10 action permit
set policy route-map RM-VOICE-OUT rule 10 match ip source address prefix-list PL-VOICE-SRC
set policy route-map RM-VOICE-OUT rule 10 set ip next-hop '198.51.100.1'

The operational smell:

• The PBR works fine until the upstream router reboots or the link fails.
• After the upstream fails, voice traffic is silently dropped. No failover to the backup provider because the PBR rule hard-codes the primary next-hop.
• ip route get 8.8.8.8 from 10.10.0.5 returns the dead next-hop; ping 198.51.100.1 fails.

The remediation:

• Use a tracking object (BFD, track) to monitor the next-hop and disable the route-map rule when the next-hop fails.
• Or, use a recursive next-hop via a different table that contains a dynamic route. The dynamic route fails over; the recursive next-hop follows.
• Or, use multiple next-hops in an ECMP group; the kernel load-balances and the dead next-hop is naturally avoided after the kernel detects it is unreachable.

Anti-pattern 7 — missing rule description and runbook entry

The operator configures the PBR rule but does not document the intent. The next operator cannot determine why the rule exists or what it does.

set policy route-map RM-XYZ rule 10 action permit
set policy route-map RM-XYZ rule 10 match ip source address prefix-list PL-A
set policy route-map RM-XYZ rule 10 set ip next-hop '203.0.113.1'

The operational smell:

• The route-map name RM-XYZ is meaningless.
• The next-hop 203.0.113.1 is not associated with a provider in the runbook.
• The change request that introduced the rule is not linked.
• The post-incident review cannot determine the operator intent.

The remediation:

• Use descriptive route-map names: RM-VOICE-OUT-PROVIDER-A, not RM-XYZ.
• Add a description to each rule that names the traffic class, the next-hop, and the change request.
• Document the rule in the team’s runbook with the source prefix, the next-hop, the operator intent, and the test commands that verify the rule is working.

How the configuration is validated

The validation command set for PBR anti-pattern detection:

show policy route-map
show ip rule
ip rule show
ip route show table all
ip route get <src> <dst>
tcpdump -ni <egress-ifname> 'src <src>'
show configuration commands | match 'policy route-map'

A clean PBR deployment has:

• Descriptive route-map names with per-rule descriptions.
• Route-maps bound only to the interfaces that carry the intended traffic class.
• Per-rule tracking on hard-coded next-hops.
• A runbook entry documenting the intent.
• A test command (ip route get) that confirms the rule fires for the intended source.

How it fails

The production failure modes the engineer must recognise:

• PBR-everywhere. The router has more PBR rules than the main table has routes. The data plane is dominated by rule-walking; the design is unmaintainable.
• Conflicting route-maps. Two route-maps bound to the same interface with overlapping source prefixes; the operator cannot predict which wins.
• Missing PBR table. The route-map is defined but not bound to an interface; the rule never fires.
• Asymmetric PBR. The forward path uses PBR; the return path uses the main table. Stateful firewalls drop the return packets.
• Hard-coded next-hop down. The PBR rule hard-codes a next-hop that has failed. The kernel still forwards to the dead next-hop.
• Missing documentation. The PBR rule has no description and no runbook entry. The next operator cannot understand the intent.

Rollback

The recovery from a PBR anti-pattern:

• PBR-everywhere: redesign the network to use VRFs and destination-based routing; remove the PBR rules that are not necessary.
• Conflicting route-maps: combine the route-maps into one with explicit sequence ordering.
• Missing table: add the interface binding.
• Asymmetric PBR: apply the PBR rule to the return path or disable stateful firewall on the affected interfaces.
• Hard-coded next-hop: add tracking (BFD, track, recursive next-hop).
• Missing documentation: add the description and the runbook entry.

Production discipline

Cross-course references

The Linux course’s XIX-Linux-NetFoundations covers the kernel FIB and the rule selector that PBR uses. The VyOS lessons vyos-xiii-01-pbr-concept, vyos-xiii-02-route-maps, vyos-xiii-03-pbr-rules, vyos-xiii-04-pbr-ipv6, and vyos-xiii-05-pbr-troubleshoot cover the PBR foundation this lesson assumes. The lessons vyos-xv-01-vrf-concept and vyos-xv-02-vrf-config cover the VRF alternative that PBR should defer to when possible. The lessons vyos-xlv-01-qos-concept and vyos-xlv-02-dscp-marking cover the QoS alternative for application-based routing. The lesson vyos-xxxi-04-route-not-installed covers the troubleshooting pattern for hard-coded next-hops that fail.

Quiz