Routing table anti-patterns — too many tables, conflicting priorities, FRR not pushing

Multiple routing tables are a powerful tool on VyOS 1.5 LTS, and like every powerful tool, they have a long list of ways to be misused. The anti-patterns fall into four broad categories: tables that exist without documentation, rules that conflict at the priority layer, tables that have no matching rule, and routes that FRR has not pushed into the kernel because of a static-table configuration error. This lesson walks through each category, shows what the failure looks like at the operator shell, and lays out the design discipline that prevents the trap.

Anti-pattern 1 — too many tables without documentation

The first anti-pattern is the proliferation of tables. An operator deploys a multi-WAN source-based routing setup with two tables (isp-a, isp-b), then a VPN overlay with a third (vpn), then a backup path with a fourth (backup), then a DMZ with a fifth (dmz), then a guest network with a sixth (guest). Each table has a meaningful purpose at the moment of creation; six months later, the next operator cannot remember which table carries which traffic.

flowchart TB
  A[Table mgmt] --> A1[ID 100]
  B[Table isp-a] --> B1[ID 200]
  C[Table isp-b] --> C1[ID 300]
  D[Table vpn] --> D1[ID 400]
  E[Table backup] --> E1[ID 500]
  F[Table dmz] --> F1[ID 600]
  G[Table guest] --> G1[ID 700]
  H[Table legacy] --> H1[ID 800]
  H --> H2[Created 2024?]
  H --> H3[Still referenced?]

The failure modes of undocumented tables:

• Stale tables with no routes. A table is created, used for a project, the project ends, the routes are deleted, the rt-table declaration is left. Six months later, the table exists in /etc/iproute2/rt_tables with no purpose. New operators cannot tell if it is safe to remove.
• Duplicate-purpose tables. Two tables that were created for the same purpose at different times by different operators. Each carries a subset of the traffic; the routing is inconsistent.
• ID collisions when adding new tables. An operator adds a new table at ID 800 without checking that 800 is free. The commit fails because a stale table is already there. The operator picks a random new ID, leaving the stale table in place.

The discipline is a documented table inventory. Every table in the configuration must have:

• A description in set system ip rt-table that names the purpose, the change request that introduced it, and the date.
• At least one rule that selects it (or a documented reason why it is a fallback).
• A corresponding set protocols static table <name> (or equivalent) with the routes the table carries.
• A deletion date in the description if it is scheduled for retirement.

Anti-pattern 2 — conflicting rule priorities

The second anti-pattern is rules that conflict at the priority layer. The RPDB walks rules in ascending priority order; the first matching rule wins. Two rules that both match the same traffic — one with a narrower selector and one with a wider selector — produce a conflict that the operator usually does not notice until traffic flows incorrectly.

flowchart TD
  A[Packet from 10.10.0.5 to 8.8.8.8] --> B[RPDB walk]
  B --> P0[Priority 0: local]
  P0 --> P1[Priority 100: from 10.10.0.0/16 lookup mgmt]
  P1 --> P2[Priority 200: from 10.10.0.0/24 lookup isp-a]
  P2 --> P3[Priority 32766: main]

In the diagram, priority 100 selects mgmt for all traffic from 10.10.0.0/16 — including the more-specific 10.10.0.0/24. The priority 200 rule never gets consulted for traffic from 10.10.0.0/24 because the priority 100 rule already matched.

The trap is that the operator believes the priority 200 rule will handle 10.10.0.0/24 traffic because the selector is more specific. The RPDB does not care about specificity; it cares about priority. The wider rule at the higher priority wins.

The discipline is a documented priority allocation. Every rule in the configuration must have:

• A unique priority in the operator-defined range (1-32765 for rules that override main, 32768+ for rules that are consulted after).
• A documented purpose in the description attribute (if available) or in the deployment runbook.
• No overlap with rules at higher priority that supersede the intended selector.

Anti-pattern 3 — table without a matching rule

The third anti-pattern is a custom table with no matching rule. The table has routes; the routes are in the kernel; the operator sees the routes in ip route show table <name>. But no ip rule selects the table for any traffic, so no packet ever consults it. The routes exist; the table is invisible.

vyos@vyos:~$ show configuration system ip rt-table
100	mgmt	management traffic

vyos@vyos:~$ show configuration system ipv4 rule
(no output)

vyos@vyos:~$ ip route show table mgmt
10.30.0.0/24 via 192.0.2.3 dev eth0 proto static metric 1

vyos@vyos:~$ ip route get 10.30.0.5 from 10.10.0.5
10.30.0.5 from 10.10.0.5 via 192.0.2.3 dev eth0 table main uid 0
    cache

The ip route get returns table main, not table mgmt. The table mgmt has the route; the walk never reaches the table because there is no rule that selects it.

The trap is that the operator sees the route and assumes it is forwarding. The route is forwarding for traffic that consults mgmt. The RPDB does not consult mgmt for any traffic because no rule says so. The route is dormant.

The discipline is to pair every rt-table declaration with a matching rule. The VyOS configuration tree does not enforce this; the operator must. A simple operational rule: every set system ip rt-table rt-table-id <id> name <name> line should have a corresponding set system ipv4 rule <priority> ... table <name> line in the same commit (or in the same deployment runbook).

Anti-pattern 4 — FRR not pushing routes to the custom table

The fourth anti-pattern is FRR’s static daemon not pushing routes to a custom table. The VyOS configuration has the route in the custom table; the commit succeeds; show configuration protocols static table <name> shows the route; but the route does not appear in ip route show table <name>.

vyos@vyos:~$ show configuration protocols static table mgmt
route 10.30.0.0/24 {
    next-hop 192.0.2.3
}

vyos@vyos:~$ ip route show table mgmt
(no output)

vyos@vyos:~$ vtysh -c 'show ip route table mgmt'
(no output)

The FRR zebra log may show:

2026/08/15 10:23:45 zebra: [EC 4043309111] RtInstall: rtm中新_route for 10.30.0.0/24 via 192.0.2.3 failed: Network is unreachable

The cause is the next-hop. The static route in the custom table references next-hop 192.0.2.3, but the egress interface eth0 has the address 192.0.2.1/24. The kernel validates the next-hop: the route can only be installed if the next-hop is reachable on a connected subnet.

For the main table, the connected route to 192.0.2.0/24 via eth0 is present. For the mgmt table, the connected route is not automatically copied — only routes explicitly installed in the table appear in the table. The kernel cannot resolve the next-hop because the mgmt table has no connected route for 192.0.2.0/24.

The fix is one of:

1. Add the connected route to the custom table. VyOS does not automatically do this. The operator must explicitly configure set protocols static table <name> route <connected-prefix> interface <iface> for every connected subnet that the table needs.
2. Use a recursive next-hop. The next-hop is itself a route in another table (e.g. the upstream ISP gateway is reachable via a route in main). The static route in the custom table installs; the kernel resolves the next-hop recursively.
3. Use a direct next-hop with explicit interface. The static route has next-hop X interface Y, where Y is the egress interface. The kernel installs the route without needing a recursive lookup.

Anti-pattern 5 — asymmetric routing with source-based egress

The fifth anti-pattern is asymmetric routing with source-based egress. The forward path is correct; the return path is routed via a different ISP; the connection is broken. This is covered in detail in xiv-04; the lesson here is the design discipline.

The discipline is: every source-based egress deployment must have a documented return-path story. The story is one of:

• BGP with the upstream. Announce the site’s prefixes with attributes that influence the upstream’s return path. This is the most common production answer.
• Source-address binding. SNAT the traffic to the egress interface address so the upstream’s return path is deterministic.
• Negotiated return routes. The upstream ISP agrees to route the site’s source addresses via the same ISP. This is rare without BGP.

Without one of these, source-based egress creates an asymmetric path that breaks stateful devices on the path. The fix is not at the local router; it is at the routing announcement.

How to audit a multi-table deployment

The audit command set:

vyos@vyos:~$ show configuration system ipv4 rule | count-lines
4

vyos@vyos:~$ show configuration system ip rt-table | count-lines
3

vyos@vyos:~$ show configuration protocols static | grep "table " | count-lines
5

The counts should match: every rt-table declaration should have at least one rule; every route configuration that references a table should match the declared tables.

A second audit is the route inventory:

vyos@vyos:~$ for table in $(ip route show table all | grep "^table"); do
    echo "Table: $table"
    ip route show table $table | wc -l
done

Every table should have at least one route. Empty tables are stale (or were created in anticipation of routes that were never added). Tables with one route are suspect (often the operator added a default but forgot the connected routes).

A third audit is the cross-reference between rules and tables:

vyos@vyos:~$ for rule_priority in $(ip rule show | awk '{print $1}' | tr -d ':'); do
    rule=$(ip rule show priority $rule_priority 2>/dev/null)
    if [[ "$rule" == *"lookup"* ]]; then
        table=$(echo "$rule" | awk '{print $NF}')
        route_count=$(ip route show table $table 2>/dev/null | wc -l)
        echo "Rule $rule_priority -> table $table -> $route_count routes"
    fi
done

The output should show: every rule references a table; every referenced table has routes; the counts match. Any anomaly — a rule with no table, a table with no routes, a count mismatch — is a finding.

How it fails

The production failure modes the engineer must recognise:

• Table proliferation without documentation. Six tables in the configuration; only two have routes; one is referenced by a rule but the rule’s selector is wrong. The deployment is unmaintainable.
• Rule priority conflict. A high-priority rule with a wide selector shadows a lower-priority rule with a narrower selector. Traffic follows the wide selector; the operator expects the narrow selector.
• Table without rule. Routes in mgmt; no rule selects mgmt; traffic to the prefix is forwarded by main; the operator believes the deployment is working.
• FRR not pushing to custom table. The route is in the configuration; the route is not in the kernel. The cause is a next-hop that the custom table cannot resolve because the connected route is not present.
• Asymmetric routing. Forward via ISP-A; return via ISP-B; TCP connections broken. The fix is upstream, not local.

Rollback

The recovery from an anti-pattern depends on the failure mode:

• Stale tables: delete system ip rt-table rt-table-id <id>; commit; save.
• Conflicting priorities: delete system ipv4 rule <priority>; set system ipv4 rule <new-priority> ...; commit; save.
• Table without rule: add the rule (set system ipv4 rule <priority> ... table <name>; commit; save) or delete the table.
• FRR not pushing: add the connected route to the custom table (set protocols static table <name> route <connected-prefix> interface <iface>; commit; save) or use a direct next-hop with explicit interface.
• Asymmetric routing: deploy BGP with the upstream or fall back to destination-based routing.

Production discipline

Cross-course references

The Linux course’s XXI-Linux-NetAdvanced covers the same anti-patterns from a host perspective. The PBR course’s XIII-VyOS-PBR covers policy-routing anti-patterns more broadly. The BGP course’s XXV-BGP-Advertisement covers the return-path discipline that prevents the asymmetric-routing trap.

Quiz