Recursive routing — indirect next-hops, MTU, and RFC 8308 path MTU

A recursive static route is a route whose next-hop is not on a directly connected subnet. The kernel cannot ARP-resolve the next-hop on the local segment; it must consult the routing table to find an egress interface that can reach the next-hop. The kernel walks the table, finds a connected route to the next-hop’s subnet, and uses that interface as the egress. The route is “indirect” — its forwarding depends on another route in the table. This lesson covers the recursive model, the configuration, the failure modes that arise when the next-hop is not directly reachable, and the MTU and path-MTU issues that complicate the picture.

What recursive routing is

flowchart TD
  A[Packet to 10.20.0.5] --> B[Lookup 10.20.0.0/16 in RIB]
  B --> C[Route: 10.20.0.0/16 via 192.0.2.99]
  C --> D[192.0.2.99 is not on a connected subnet]
  D --> E[Recursive lookup: how do I reach 192.0.2.99?]
  E --> F[Lookup 192.0.2.99 in RIB]
  F --> G[Connected route: 192.0.2.0/24 via eth0]
  G --> H[Egress interface: eth0]
  H --> I[ARP-resolve 192.0.2.99 on eth0]
  I --> J[Forward the packet]

A recursive static route is a route where the next-hop is not on a directly connected subnet. The route’s egress interface is not stated in the route; the kernel must compute it by looking up the next-hop in the routing table.

For an Ethernet segment where the next-hop is on the same subnet as the local interface, the lookup is trivial — the kernel finds a connected route to the next-hop’s subnet and uses the corresponding interface. The route is installed “active” in the FIB.

For a transit topology where the next-hop is on a different subnet, the lookup is more complex. The kernel walks the table and may find a static, OSPF, or BGP route that describes how to reach the next-hop’s subnet. The kernel uses that route’s egress interface as the egress for the recursive route. The route is “active” if and only if the recursive lookup resolves.

The configuration idiom

A recursive static route is a route with a next-hop but no egress interface. The kernel resolves the egress through the recursive lookup:

[edit]
vyos@vyos# set protocols static route 10.20.0.0/16 next-hop 192.0.2.99
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# save

The next-hop is 192.0.2.99. The egress interface is not stated. The kernel looks up 192.0.2.99 in the table and finds the connected route 192.0.2.0/24 via eth0. The recursive route uses eth0 as the egress.

show ip route shows the recursive resolution:

vyos@vyos:~$ show ip route 10.20.0.0/16
S>* 10.20.0.0/16 [1/0] via 192.0.2.99, eth0

The via 192.0.2.99 is the configured next-hop. The , eth0 is the kernel’s resolved egress. The * confirms the route is in the FIB.

A direct route (next-hop + interface) is the alternative:

[edit]
vyos@vyos# set protocols static route 10.20.0.0/16 next-hop 192.0.2.99 interface eth0
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# save

The kernel installs the route with the explicit egress. No recursive lookup is required; the route is “active” as soon as the egress interface is up.

The choice between recursive and direct is operational. The direct form removes a lookup from the hot path; the recursive form is more flexible because the egress is determined by the table. In a transit topology with multiple paths to the next-hop, the recursive form benefits from the table’s selection; the direct form is pinned to a single interface.

The inactive state

The recursive route is inactive when the next-hop is not in the table. The cause is usually a missing connected route — the egress interface has no address, or the address is on the wrong subnet.

vyos@vyos:~$ show ip route 10.20.0.0/16
S   10.20.0.0/16 [1/0] via 192.0.2.99 inactive

The route is in the RIB (the S is present) but not in the FIB (no *). The reason is inactive — the next-hop 192.0.2.99 is not in the table.

The fix is to ensure the next-hop is reachable. The most common cause is a missing or wrong address on the egress interface:

vyos@vyos:~$ show ip route 192.0.2.0/24

If the connected route is absent, the egress interface has no address on 192.0.2.0/24. The recursive route has no next-hop reachability; it is inactive.

The fix is to add the address:

[edit]
vyos@vyos# set interfaces ethernet eth0 address 192.0.2.1/24
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# save

After the commit, the connected route appears, the recursive route becomes active, and the * appears in the FIB.

MTU issues in recursive topologies

Recursive routing introduces MTU issues that are not present in direct routing. The MTU of the egress interface is the MTU of the local segment, but the path MTU of the full route may be lower.

A typical topology:

• Local router R1 with a recursive route to 10.20.0.0/16 via next-hop 192.0.2.99.
• Transit router R2 with a connected route to 192.0.2.0/24 on the R1-R2 link.
• Destination 10.20.0.5 on a remote segment reached through R2.

R1’s egress MTU to R2 is 1500 (standard Ethernet). R2’s egress MTU to 10.20.0.0/16 is 1500. The end-to-end MTU is 1500. No issues.

But if R2’s egress is a tunnel (e.g. GRE, IPsec, VXLAN), the tunnel adds overhead. The effective MTU of the tunnel is lower. R1 sends a 1500-byte packet to R2; R2 encapsulates the packet in the tunnel; the encapsulated packet exceeds the underlying MTU; the packet is dropped or fragmented.

Diagnosing MTU issues

The standard MTU test is ping with the don't fragment flag and a specific size:

vyos@vyos:~$ ping -M do -s 1472 -c 3 10.20.0.5
PING 10.20.0.5 (10.20.0.5) 1472(1500) bytes of data.
--- 10.20.0.5 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss

The -M do sets the DF flag. The -s 1472 sets the payload size to 1472 (total packet 1500 with IP+ICMP headers). The test confirms the path MTU is 1500 or larger.

A test with a larger size that fails:

vyos@vyos:~$ ping -M do -s 8972 -c 3 10.20.0.5
PING 10.20.0.5 (10.20.0.5) 8972(9000) bytes of data.
--- 10.20.0.5 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss

The 9000-byte packet is dropped because the path does not support jumbo frames. The standard ping test confirms the MTU.

The ICMP evidence:

vyos@vyos:~$ tcpdump -ni any icmp
13:45:23.456 IP 10.0.0.1 > 10.20.0.5: ICMP echo request, id 1, length 9000
13:45:23.456 IP 192.0.2.99 > 10.0.0.1: ICMP 192.0.2.99 unreachable - need to frag (mtu 1500)

The 192.0.2.99 unreachable - need to frag is the path-MTU-Discovery signal. The source’s MTU is reduced; the next ping uses a smaller size.

If the ICMP is missing, the firewall is dropping the path-MTU signal. The source’s MTU stays at 9000; large packets are silently dropped; PMTUD fails.

RFC 8308 path MTU discovery

RFC 8308 (Path MTU Discovery for IP version 6) and the related IPv4 standardisation in RFC 1191 define a more robust PMTUD mechanism. The mechanism is the same in principle: source sends DF packet, router sends ICMP unreachable, source reduces MTU.

The VyOS 1.5 LTS Linux kernel supports the standard PMTUD mechanism. The relevant sysctl is net.ipv4.ip_no_pmtu_disc, which is 0 (PMTUD enabled) by default.

vyos@vyos:~$ sysctl net.ipv4.ip_no_pmtu_disc
net.ipv4.ip_no_pmtu_disc = 0

If the operator has disabled PMTUD (= 1), the kernel sends large packets that may be silently dropped on the path. The fix is to re-enable PMTUD:

[edit]
vyos@vyos# set system sysctl parameter net.ipv4.ip_no_pmtu_disc value 0
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# save

For IPv6, PMTUD is mandatory; the kernel does not fragment IPv6 packets. The relevant sysctl is net.ipv6.ip6_no_pmtu_disc, which is 0 (PMTUD enabled) by default and should not be changed.

How the result is validated

The validation command set confirms the recursive route is active and reachable:

show ip route <prefix>
ip route get <prefix>
ping -M do -s <size> <destination>
tcpdump -ni any icmp

The first shows the route and its resolved egress. The second shows the kernel’s lookup result for a specific destination, including the egress interface and the next-hop chain. The third confirms the path MTU. The fourth shows the ICMP evidence when PMTUD is exercised.

ip route get is the canonical command for a recursive route:

vyos@vyos:~$ ip route get 10.20.0.5
10.20.0.5 via 192.0.2.99 dev eth0 src 192.0.2.1 uid 0
    cache

The output shows the egress interface (eth0), the next-hop (192.0.2.99), and the source address (192.0.2.1). The cache means the kernel has cached the result of the recursive lookup.

How it fails

The production failure modes the engineer must recognise:

• Inactive recursive route. The next-hop is not in the table. The route is inactive; traffic is dropped or forwarded to a less-specific route. The most common cause is a missing or wrong interface address.
• Recursive next-hop to wrong subnet. The next-hop is in the table but on the wrong subnet. The kernel resolves the egress to a different interface. The packet is forwarded to the wrong segment; the destination never sees it.
• MTU mismatch on the recursive path. A tunnel or encapsulation with a lower MTU. Large packets are silently dropped; PMTUD may or may not recover.
• PMTUD blocked by firewall. A firewall that drops ICMP-unreachable. PMTUD fails; large packets are dropped; the connection is broken.
• Recursive chain too long. A next-hop whose egress is another recursive route. The chain can be many levels deep; the kernel has a limit (typically 8 levels). A routing loop or a poorly-designed topology can hit the limit.

Rollback

The recovery from a bad recursive route:

• Wrong next-hop: set protocols static route <prefix> next-hop <correct>; commit; save.
• Add the interface for a direct form: set protocols static route <prefix> next-hop <ip> interface <ifname>; commit; save.
• Fix the egress address: set interfaces ethernet <ifname> address <correct>; commit; save.
• Whole-tree rollback: rollback N; commit; save.

Production discipline

Cross-course references

The MTU course’s LI-MTU covers path MTU discovery in detail. The VPN course’s XLIII-VPN-Routing covers MTU issues in tunneled topologies. The Linux course’s V-Linux-NetConfig covers the ip route get and PMTUD mechanisms. The IPsec course’s XLII-IPsec covers the MTU overhead for IPsec tunnels.

Quiz