The routing-table concept — RIB, FIB, FRR, and the Linux kernel datapath

Every packet a VyOS router forwards has been looked up in a forwarding information base (FIB). Every route the operator configures has been stored in a routing information base (RIB). The two structures sound the same and they live on the same machine, but they are different in purpose, in lifetime, and in who populates them. This lesson covers the concept: what a routing table is on VyOS 1.5 LTS, how the RIB feeds the FIB, what FRRouting’s zebra does in between, and why table 254 is the default the kernel uses when no policy rule intervenes.

The RIB and the FIB are different things

flowchart LR
  A[Operator set ...] --> B[VyOS commit]
  B --> C[FRR zebra RIB]
  C -->|netlink RTM_NEWROUTE| D[Kernel RIB]
  D --> E[Kernel FIB]
  E -->|per-packet lookup| F[Forwarding decision]

The RIB is the source of truth for routes. It is owned by FRR’s zebra daemon on a VyOS router. Zebra stores the routes the operator configured, the routes the routing protocols learned, and the connected routes that the kernel told it about. Zebra’s RIB is in memory; it is not persisted to disk; it is rebuilt on every boot from the saved configuration and the protocol state.

The FIB is the data-plane lookup structure. It is owned by the Linux kernel. The FIB is what the kernel walks for every forwarded packet. The FIB is the hot path; it must be small, fast, and resident in memory. The kernel builds the FIB from the RIB through the netlink socket.

A route can be in the RIB but not in the FIB (FRR learned the route, the kernel rejected the install) or in the FIB but not in the RIB (the kernel added a connected or a kernel-side default). The two views should agree, but they don’t always.

FRRouting’s role

VyOS does not implement its own routing engine. It implements a configuration front-end and a commit engine that renders configuration into FRR’s syntax. FRR is the daemon that actually runs the routing protocols and maintains the RIB.

flowchart TB
  subgraph FRR[FRRouting]
    Z[zebra - RIB owner]
    BG[bgpd]
    OS[ospfd]
    IS[isisd]
    ST[staticd]
  end
  subgraph Linux[Linux kernel]
    R[RIB via netlink]
    F[FIB - data plane]
  end
  Z -->|netlink RTM_NEWROUTE| R
  BG --> Z
  OS --> Z
  IS --> Z
  ST --> Z
  R --> F

Each routing protocol in FRR — bgpd, ospfd, isisd, staticd — learns its own routes and passes them to zebra. Zebra runs the best-path selection for the local router and resolves the table. Once zebra has decided which route wins for each prefix, it pushes the route into the kernel through the netlink socket using RTM_NEWROUTE messages.

On VyOS 1.5 LTS, the static daemon is staticd, the OSPF daemon is ospfd, the BGP daemon is bgpd. Zebra is the single converger: protocols do not install routes into the kernel directly; they install them into zebra, which installs them into the kernel.

The Linux kernel netnext FIB

The Linux kernel forwarding implementation lives under net/ipv4/fib_trie.c (and the IPv6 equivalent). The structure is an LC-trie (level-compressed trie) keyed on the destination prefix. The trie supports longest-prefix-match lookup in O(k) time where k is the address length in bits (32 for IPv4, 128 for IPv6).

flowchart TB
  A[Packet to 10.20.0.5] --> B[FIB lookup]
  B --> C{Trie walk}
  C -->|/16 hit| D[10.20.0.0/16 via 192.0.2.2]
  C -->|/24 miss| E[Try shorter prefix]
  E -->|/8 miss| F[Default route 0.0.0.0/0]
  F -->|hit| G[Forward via 192.0.2.1]
  D -->|hit| H[Forward]

The FIB is per-table. The kernel supports up to 2^32 routing tables per address family (in practice, the ID space is 0 to RT_TABLE_MAX defined in the kernel headers). Each table has its own trie. The lookup walks the trie for the table the routing policy database (RPDB) selected for this packet.

The netnext FIB refers to the upstream Linux networking tree where new routing features land. VyOS 1.5 LTS runs on Debian Bookworm (Linux 6.1 LTS at release), which includes the netnext FIB improvements: lockless lookups (fib_lookup RCU), per-CPU caches for the FIB, nexthop exception objects for fast-path ECMP, and per-table hash limits to bound the FIB size. The operator does not configure these directly; they are kernel internals. They matter because they are why a VyOS router can forward millions of packets per second on commodity hardware.

Default table 254

When no ip rule selects a different table, the kernel uses table 254 — the main table. The kernel ships with three tables pre-declared:

The IDs 0, 253, 254, and 255 are reserved. Operator-defined tables must use IDs outside this range. The convention is to use IDs from 1 upward, with a documented mapping in /etc/iproute2/rt_tables.

vyos@vyos:~$ cat /etc/iproute2/rt_tables
#
# reserved values
#
255	local
254	main
253	default
0	unspec
#
# local
#
1	inr.ruhep

The file maps numeric IDs to names. The kernel does not read this file; the file is read by iproute2 to translate names like main into numeric IDs. A custom table named mgmt at ID 100 would be declared here so ip route show table mgmt works.

How the result is validated

The validation command set is layered. Each layer shows a different view of the same route.

vyos@vyos:~$ show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - table, v - VNC, V - VNC-Direct, A - Babel,
       D - SHARP, F - PBR, f - OpenFabric,
       > - selected route, * - FIB route

S>* 10.20.0.0/16 [1/0] via 192.0.2.2, eth0, 00:00:12
C>* 192.0.2.0/24 is directly connected, eth0

The show ip route output is FRR’s view of the RIB. The * means the route is in the FIB. The > means it is the selected route for the prefix. Both columns should be populated for a route that is forwarding traffic.

The ip route show output is the kernel’s view of the FIB:

vyos@vyos:~$ ip route show
10.20.0.0/16 via 192.0.2.2 dev eth0 proto static metric 1
192.0.2.0/24 dev eth0 proto kernel scope link src 192.0.2.1

The proto static confirms the route was installed by the static routing source. The metric 1 is the kernel’s view of the distance. The two outputs should agree. If they do not, the route is in FRR but not in the kernel, and traffic to the prefix falls back to a less-specific route or to the default.

The ip route show table all command lists the routes in every table, not just the main table:

vyos@vyos:~$ ip route show table all
table 254:
10.20.0.0/16 via 192.0.2.2 dev eth0 proto static metric 1
192.0.2.0/24 dev eth0 proto kernel scope link src 192.0.2.1
table local:
local 192.0.2.1 dev lo proto kernel scope host src 192.0.2.1
broadcast 192.0.2.255 dev lo proto kernel scope link src 192.0.2.1

The default is to show table 254 (main). The table all form shows every table. This is the command that surfaces unreferenced tables — tables that have routes but no ip rule that selects them. Those routes exist but are never consulted.

How it fails

The production failure modes the engineer must recognise:

• Route in FRR but not in the kernel. A zebra netlink error or a kernel-side rejection (e.g. an unreachable next-hop with strict kernel validation). The show ip route shows the route; ip route show does not. Check /var/log/frr/zebra.log for the netlink error.
• Route in kernel but not in FRR. The kernel added a connected, a kernel-side default, or a route installed by another tool (e.g. ip route add from a script). The operator did not write the route through VyOS; it is invisible to FRR.
• Route in a table with no rule. A static route in table 100 with no ip rule add from ... table 100. The route exists; no packet consults the table. The kernel falls back to the next rule or to the default.
• Default table changed. An operator script that issues ip route add default via ... table 254 outside of VyOS. The default route in main is not what VyOS configured. After a reboot or commit, the script-installed route is gone.
• Table ID conflict with VRF. A custom table at an ID that collides with a VRF table. The kernel does not allow the same ID for a table and a VRF; the ip rule add returns EEXIST.

Rollback

The recovery from a routing-table issue depends on the failure mode. The canonical rollbacks:

• Wrong route in main: delete protocols static route <prefix>; commit; save.
• Custom table misconfigured: delete system ip rt-table <name>; commit; save (removes the alias and any rules that reference it).
• Stale ip rule from a script: delete system ipv4 rule <spec>; commit; save.
• Whole-tree rollback: rollback N; commit; save.

For emergency rollback, load <file>; commit; save replaces the candidate with a saved backup. For live-state rollback (a script that bypassed VyOS), the operator must remove the offending route with ip route del ... and re-apply the correct configuration through VyOS.

Production discipline

Cross-course references

The Linux course’s XXI-Linux-NetAdvanced covers the kernel side of multiple routing tables — the same primitives from the host perspective. The OPNsense course’s II-OPNsense-Routing covers the FreeBSD FIB and setfib equivalents. The BGP course’s XXV-BGP-Advertisement covers the protocol-side of route installation; the FIB/RIB split in this lesson is what makes that installation visible.

Quiz