Route preference — administrative distance, longest-prefix-match, and which route wins

A production router has, for almost every prefix it knows about, multiple candidate routes from multiple sources. A default route from the upstream BGP. A static route configured by the operator. An OSPF-learned route from inside the AS. A BGP-learned route from a backup transit provider. For any one prefix the router installs exactly one of these into the kernel — it will not install three next-hops for the same prefix without explicit multipath policy — and the discipline that decides which one it installs is the discipline of route preference.

This lesson is about how FRR makes that decision, the VyOS 1.5 configuration that overrides it, and the production failure modes that come from overriding the wrong thing.

Two different questions that get confused

Operators routinely collapse two separate mechanisms into one mental model, and then tune the wrong one. Keep them apart:

• Route selection happens in FRR’s RIB, per prefix, at the moment routes change. Among all candidate routes for the same prefix, zebra picks one and installs it. This is where administrative distance and metric live.
• Forwarding lookup happens in the kernel, per packet, against all the prefixes that are already installed. Longest-prefix-match happens here, between different prefixes, long after selection is over.

A /24 does not “beat” a /16. They are not competing: both are installed, and a packet simply matches whichever is more specific. Administrative distance never enters that comparison, and no distance you configure on the /16 will make it carry traffic destined inside the /24.

flowchart TD
  A[Candidate routes arrive] --> B{Group by exact prefix}
  B --> C[Contest within 10.0.0.0/8]
  B --> D[Contest within 10.20.5.0/24]
  C --> E{Lowest administrative distance}
  E --> F{Tie? lowest metric}
  F --> G[One winner installed for /8]
  D --> H[One winner installed for /24]
  G --> I[FIB holds both prefixes]
  H --> I
  I --> J[Per packet: longest-prefix-match picks one]

The selection contest is deterministic: given the same candidates, the same route wins every time. Getting the layer right is most of the skill — the operator who tunes distance when the problem is prefix length gets a router that looks correctly configured and forwards traffic to the wrong place.

Administrative distance — the canonical defaults

Administrative distance is a router-local value that ranks routing sources. It is never advertised to a neighbour; it exists only inside this router’s RIB. Lower is more preferred. The FRR defaults VyOS inherits:

Connected wins over static; static wins over OSPF; OSPF wins over iBGP. eBGP sitting below OSPF is the deliberate part: an external route learned at the AS boundary is meant to outrank an IGP that has no first-hand knowledge of the destination.

Reading the selection out of FRR is the fastest way to check your model against the router’s:

$ show ip route 10.0.0.0/8

Routing entry for 10.0.0.0/8
Known via "bgp", distance 20, metric 0, best
Last update 00:02:11 ago
* 198.51.100.1, via eth1, weight 1

Routing entry for 10.0.0.0/8
Known via "ospf", distance 110, metric 10
Last update 00:14:02 ago
  192.0.2.2, via eth0, weight 1

Both entries are for the same /8, so this is a genuine contest. The BGP entry carries best and its next-hop carries the *; the OSPF entry has neither. BGP at distance 20 beat OSPF at distance 110, and the metrics were never consulted because the distances were not tied.

The same selection appears in the whole-table view, where the codes are compressed into flags:

$ show ip route

Codes: K - kernel route, C - connected, S - static, R - RIP,
     O - OSPF, I - IS-IS, B - BGP, > - selected route, * - FIB route

B>* 10.0.0.0/8 [20/0] via 198.51.100.1, eth1, 00:02:11
O   10.0.0.0/8 [110/10] via 192.0.2.2, eth0, 00:14:02
C>* 192.0.2.0/30 is directly connected, eth0, 01:20:33

[20/0] is [distance/metric]. The > means zebra selected this route; the * means it made it into the kernel FIB. A route with > but no * was selected and then rejected on installation, which is nearly always an unresolvable next-hop.

Configuring administrative distance in VyOS 1.5

Every protocol exposes distance differently, and the shape of each tree tells you what the knob can and cannot scope to.

Static routes take distance per next-hop, which is the finest granularity available anywhere:

[edit]
vyos@vyos# set protocols static route 10.20.0.0/16 next-hop 192.0.2.2
[edit]
vyos@vyos# set protocols static route 10.20.0.0/16 next-hop 198.51.100.1 distance 210
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# save

The first next-hop keeps the default distance of 1 and is the primary. The second at distance 210 is a floating static: it sits in the RIB as a candidate and only becomes the selected route when nothing better — including the distance-1 static above it — is usable. That is the canonical primary / backup pattern, and it works because both entries are the same prefix and therefore actually compete.

OSPF takes a process-wide distance:

[edit]
vyos@vyos# set protocols ospf distance global 90
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# save

distance global 90 moves every OSPF route from 110 to 90. Read what that actually buys before assuming it solved anything: 90 now beats IS-IS at 115, RIP at 120 and iBGP at 200, and still loses to eBGP at 20 and to any static at 1. The number is the whole argument — an override that does not cross the distance of the source you were competing with changes nothing except the output of show ip route. FRR additionally distinguishes intra-area, inter-area and external OSPF routes, and VyOS surfaces those as sibling leaves under the same distance node; confirm the exact leaf names on your release with the CLI’s own completion (set protocols ospf distance followed by the completion key) before you script them.

BGP takes distance per address-family, not per neighbour:

[edit]
vyos@vyos# set protocols bgp address-family ipv4-unicast distance global external 20
[edit]
vyos@vyos# set protocols bgp address-family ipv4-unicast distance global internal 200
[edit]
vyos@vyos# set protocols bgp address-family ipv4-unicast distance global local 200
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# save

Metric as the tie-breaker

Metric is consulted only when two candidates for the same prefix have the same distance — which in practice means they came from the same protocol. For OSPF the metric is the accumulated path cost; for BGP it is MED; for IS-IS it is the wide metric.

$ show ip route 10.0.0.0/8

Routing entry for 10.0.0.0/8
Known via "ospf", distance 110, metric 20, best
Last update 00:05:44 ago
* 192.0.2.2, via eth0, weight 1

Routing entry for 10.0.0.0/8
Known via "ospf", distance 110, metric 30
Last update 00:05:44 ago
  192.0.2.3, via eth1, weight 1

Both are OSPF at distance 110, so the tie falls to metric and 20 beats 30. Had the metrics also been equal, OSPF would have offered both next-hops as equal-cost paths and zebra would have installed a multipath route.

The OSPF metric on a path is the sum of the outbound interface costs along it. The default cost of an interface is derived from its bandwidth against a reference of 100 Mbit/s, so anything at or above 100 Mbit/s lands on the floor of cost 1 — which is why a 10 Gbit/s backbone link and a 100 Mbit/s access link are indistinguishable to OSPF until somebody intervenes. The two knobs are set protocols ospf interface eth0 cost 50 for a single link and set protocols ospf auto-cost reference-bandwidth 100000 to move the reference for the whole process. The next lesson covers the design consequences of both.

Longest-prefix-match — a different layer entirely

Once selection is finished, the FIB holds one route per prefix, and the kernel matches each packet against the most specific prefix that covers its destination. Nothing about distance or metric participates:

$ show ip route

S>* 10.20.0.0/16 [1/0] via 192.0.2.2, eth0, 00:31:07
S>* 10.20.5.0/24 [1/0] via 198.51.100.1, eth1, 00:30:52

Both routes are selected; both are in the FIB. A packet to 10.20.5.7 leaves via eth1 because the /24 covers it; a packet to 10.20.99.1 leaves via eth0 because only the /16 does. This is not a contest anybody won.

That distinction produces one of the most common backup-design mistakes in production. An operator configures a primary /24 and a backup as a /16, expecting the /16 to take over when the primary fails. It cannot. As long as the /24 remains in the FIB — and a static /24 stays in the FIB whenever its next-hop is still resolvable, even if the device beyond it is dead — every packet to that /24 keeps matching it and keeps being sent into the failure. The /16 is a cover for the rest of the range, not a failover for the /24.

The fix is to make the backup compete: give it the same prefix length as the primary and a higher distance, so both are candidates for the same contest and the RIB can switch between them. That is exactly the floating-static configuration shown earlier.

How the result is validated

Start with the per-source counts, which catch a whole source having gone missing before you go hunting a single prefix:

$ show ip route summary

Route Source         Routes               FIB  (vrf default)
kernel               2                    2
connected            5                    5
static               10                   10
ospf                 80                   80
ebgp                 902                  902
------
Totals               999                  999

The FIB column is the one that matters. A source with routes in the RIB but a much smaller FIB count has candidates that lost their contests or failed to install — expected if you have just added floating statics, a red flag if you have not.

Then compare FRR’s view against the kernel’s for the specific prefix:

$ ip route show 10.0.0.0/8

10.0.0.0/8 via 198.51.100.1 dev eth1 proto bgp

The two views answer different questions and it is worth being precise about how. FRR’s RIB holds every candidate along with the distance and metric that decided between them. The kernel holds only the winner, tagged with proto bgp to record which daemon put it there — the administrative distance is not in the kernel at all, because it was only ever an input to a decision FRR already made. If the kernel’s next-hop disagrees with FRR’s best, you have a control-plane / data-plane divergence rather than a preference problem, and that is the subject of the earlier lesson on convergence.

How it fails

The production failure modes the engineer must recognise:

• A floating static parked at the wrong number. 210 is above every default in the table, so the backup stays dormant until everything else is gone. Set it to 150 instead and it now outranks iBGP at 200 — so the moment an iBGP-learned route for that prefix appears, the “backup” quietly becomes the primary. Check the intended distance against the whole table, not just against the one protocol you were thinking about.
• Longest-prefix-match defeating the backup. The /24 primary and /16 backup case above. The symptom is distinctive: show ip route looks entirely healthy, both routes are selected and in the FIB, and traffic still blackholes. Whenever a failover “did not fire” but the routing table looks correct, check whether the two routes were ever the same prefix.
• A next-hop that resolves to nothing useful. A static route whose next-hop is reachable via a default route will install and stay installed even when the destination behind it is unreachable, because the RIB only checks that the next-hop resolves. This is what keeps a dead primary in the FIB, and it is why link-state failover needs BFD (set protocols static route 10.20.0.0/16 next-hop 192.0.2.2 bfd) rather than distance alone.
• A distance override with a blast radius nobody counted. set protocols ospf distance global 90 is not scoped to a prefix, an interface or an area. It applies to every OSPF route the process will ever learn, including ones nobody was thinking about when the change was made, and it survives reboots silently. The operator six months later, wondering why OSPF is winning a contest it should lose, has to find it in show configuration — which is why the comment above is not optional politeness.

Production discipline

Additional discipline:

• Give floating statics the same prefix length as the route they are backing up, and check the chosen distance against the full defaults table rather than against one protocol.
• Pair a floating static with BFD where the failure you are guarding against is a dead peer behind a live link. Distance chooses between candidates; it does not detect that the primary has stopped working.
• Express per-peer BGP preference with LOCAL_PREF, per-source preference with distance, and per-path preference with metric. Reaching for the wrong one of those three is the single most common cause of a “the configuration is right but the traffic is wrong” ticket.
• Bound every BGP peer with set protocols bgp neighbor <ip> address-family ipv4-unicast maximum-prefix <n>. A peer that leaks a full table into a session you expected 50 prefixes from will win an enormous number of contests before anybody notices.
• Use separate routing tables (set protocols static table <n> route ..., reached from policy) when two sets of preference rules genuinely conflict, rather than fighting them out with distance in one table. Part XIV covers the discipline.

Cross-course references

The static routing course’s XII-VyOS-StaticRouting covers the floating-static pattern in depth. The OSPF course’s XVIII-VyOS-OSPFFund covers OSPF cost and tie-breaking. The BGP course’s XXVII-VyOS-BGPBestPath covers the BGP best-path algorithm, where LOCAL_PREF, AS_PATH and MED are compared long before the resulting route ever meets administrative distance. The PBR course’s XIII-VyOS-PBR covers policy-based overrides for the cases preference cannot express. The multiple routing tables course’s XIV-VyOS-RoutingTables covers separating conflicting preference domains.

Quiz