Route leaking between VRFs — concept, RFC 4364, shared services

VRFs are the answer to “I need two networks on one router that cannot see each other”. Production reality is the next sentence: “but they both need the same DNS, the same NTP, and the same RADIUS.” That sentence is what route leaking exists to resolve.

This lesson is the concept: what a leak is at the routing-table level, why a well-designed estate needs one, which mechanisms VyOS 1.5 LTS actually exposes, and where the RFC 4364 reference model does and does not apply. It ends on the failure mode that justifies giving leaking a part of its own — a leak that commits cleanly, appears in the configuration, reads correctly to a reviewer, and installs no route at all.

The reference estate for this part

Every lesson in Part XVI uses the same single VyOS 1.5 router.

The shared services are not on a segment this router owns. eth0 is a transit link to the service router at 192.0.2.1, and behind it sit DNS on 198.51.100.53, NTP on 198.51.100.123 and syslog on 198.51.100.200 — the 198.51.100.0/24 block. The default context reaches all three through one static route.

configure
set vrf name mgmt table 1001
set vrf name tenant-a table 1002

set interfaces ethernet eth0 address 192.0.2.2/30
set interfaces ethernet eth1 vrf mgmt
set interfaces ethernet eth1 address 10.10.0.1/24
set interfaces ethernet eth2 vrf tenant-a
set interfaces ethernet eth2 address 10.20.0.1/24

set protocols static route 198.51.100.0/24 next-hop 192.0.2.1
commit
save

What a VRF is in the VyOS model

set vrf name mgmt table 1001 creates a Linux l3mdev VRF device and binds kernel routing table 1001 to it. Every interface enslaved with set interfaces ethernet eth1 vrf mgmt looks its destinations up in table 1001 and nowhere else. FRR runs a routing instance per context on top of that.

flowchart LR
  subgraph Global["default context"]
    GW1[eth0]
    Tbl1["table 254"]
  end
  subgraph MGMT["vrf mgmt"]
    GM1[eth1]
    TMgmt["table 1001"]
  end
  subgraph Tenant["vrf tenant-a"]
    GT1[eth2]
    TTnt["table 1002"]
  end
  GW1 --> Tbl1
  GM1 --> TMgmt
  GT1 --> TTnt

The three tables are independent. A packet entering eth1 is resolved in table 1001; a packet entering eth2 in table 1002. Neither table knows about the other, and a VRF miss does not fall through to the main table — it fails. That isolation is the point, and it is enforced by the kernel’s routing-policy rules, not by anything in the configuration you can forget to write.

Why production cannot live with pure isolation

The pure-isolation model breaks the first time a host on mgmt needs to resolve a name served from the shared segment. The resolver sends a UDP packet to 198.51.100.53; the lookup happens in table 1001 and finds no route. The packet is dropped, and the host cannot resolve anything. The same holds for NTP, RADIUS, syslog, monitoring probes, the package mirror, and the host’s ability to phone home for updates.

flowchart TB
  subgraph Global["default context"]
    DNS[DNS 198.51.100.53]
    NTP[NTP 198.51.100.123]
    LOG[syslog 198.51.100.200]
  end
  subgraph MGMT["vrf mgmt"]
    H1[host 10.10.0.10]
  end
  subgraph Tenant["vrf tenant-a"]
    H2[host 10.20.0.10]
  end
  H1 -. "no route to 198.51.100.53" .-> DNS
  H2 -. "no route to 198.51.100.123" .-> NTP
  H1 -. "no route to 198.51.100.200" .-> LOG

Route leaking is the controlled exception. The operator names the prefixes that must cross, and only those cross. Every other prefix keeps the default isolation.

The shared-services use case

The canonical production driver is shared services. A typical mid-sized estate has:

• DNS — recursive resolver, needed from every context.
• NTP — time source, needed from every context; a router whose tenants cannot reach time will produce log timestamps nobody can correlate.
• RADIUS — authentication for VPN and 802.1X.
• Syslog collector — sometimes per-VRF, with a leak to a shared aggregator.
• Monitoring — the poller has to reach the hosts, which is the reverse direction and a separate leak.
• Patch origin — internal Apt mirror, container registry, RPM repo.
• Backup target — the host that pulls nightly snapshots.

None of these belong inside a tenant VRF. All of them have to be reachable from one. Leaking each service prefix from the default context into every consumer VRF is the standard pattern, and the monitoring case is the standard reminder that direction is a design decision rather than a detail.

flowchart LR
  subgraph Global["default context - shared services"]
    S1[DNS 198.51.100.53/32]
    S2[NTP 198.51.100.123/32]
    S4[syslog 198.51.100.200/32]
  end
  subgraph MgmtA["vrf mgmt"]
    M1[hosts 10.10.0.0/24]
  end
  subgraph TenA["vrf tenant-a"]
    T1[hosts 10.20.0.0/24]
  end
  S1 -. leak .-> M1
  S2 -. leak .-> M1
  S4 -. leak .-> M1
  S1 -. leak .-> T1
  S2 -. leak .-> T1
  S4 -. leak .-> T1
  M1 -. reverse leak .-> Global
  T1 -. reverse leak .-> Global

Notice that mgmt and tenant-a never leak to each other. Only the shared segment is reachable from both, and each consumer’s return path is its own separate statement. The policy is deliberately asymmetric.

What a leak actually is

Here is the sentence the rest of this part depends on.

A route entry has two independent parts: the prefix, which decides which table the route is installed in, and the next hop, which has to be resolved somewhere. Ordinarily both belong to the same routing context. A leak is the act of separating them — install the route here, resolve its next hop there.

That is why the intuitive command does nothing:

# Commits. Reads correctly. Installs nothing.
set vrf name mgmt protocols static route 198.51.100.53/32 next-hop 192.0.2.1

FRR resolves 192.0.2.1 in the context the route belongs to, which is mgmt. Table 1001 holds the tenant’s connected route and nothing else. The next hop does not resolve, the route is held inactive, and zebra never offers it to the kernel. Nothing errors, and show configuration commands shows exactly what the operator intended.

The working form carries a vrf value on the next hop:

# Resolve 192.0.2.1 in the default context, install the result in mgmt
set vrf name mgmt protocols static route 198.51.100.53/32 next-hop 192.0.2.1 vrf default

which renders into FRR’s configuration as:

vrf mgmt
 ip route 198.51.100.53/32 192.0.2.1 nexthop-vrf default
exit-vrf

nexthop-vrf is the whole mechanism. The vrf leaf sits under both next-hop and interface in the static-route tree, so the same idea has two spellings: name an address when the destination is behind a router, and name an interface when the far side is directly connected in the other context and there is no address that would mean anything. Lesson 2 works through both.

The mechanisms VyOS 1.5 actually exposes

Earlier material in this course described three “flavours” of leaking — static, BGP, and policy. That framing is wrong in a way worth correcting, because it presents a filter as if it were a transport. Policy is a modifier applied to one of the real mechanisms; it never moves a route by itself.

Filtering is orthogonal to all three. A prefix-list and a route-map constrain what a mechanism carries; neither is a mechanism.

The RFC 4364 reference model

RFC 4364, “BGP/MPLS IP VPNs”, is the canonical reference for multi-tenant routing at the network layer. It introduced the Route Distinguisher (RD) and the Route Target (RT) extended community, and it is what service-provider L3VPN designs are built on.

Two VRFs on the same VyOS box cannot collide on 10.0.0.0/24 for a much simpler reason than the RD: their tables are separate kernel objects. The RFC 4364 model is what you reach for when the VRFs live on different routers that exchange routes over a shared underlay. The in-router mechanisms are what you reach for when they live in one box.

flowchart LR
  subgraph Local["Single chassis - VyOS 1.5"]
    L1["vrf mgmt<br/>table 1001"]
    L3["default<br/>table 254"]
    L2["vrf tenant-a<br/>table 1002"]
    L3 -. "nexthop-vrf or import vrf" .-> L1
    L3 -. "nexthop-vrf or import vrf" .-> L2
  end
  subgraph MPLS["RFC 4364 - provider edge"]
    M1["PE-A<br/>vrf RED"] --> XBGP["BGP ipv4-vpn<br/>RD + RT"]
    XBGP --> M2["PE-B<br/>vrf RED<br/>imports RT 65000:100"]
    XBGP -. "RT does not match" .-> M3["PE-B<br/>vrf BLUE<br/>imports RT 65000:200"]
  end

How traffic actually flows across a leak

A host in mgmt (10.10.0.10) sends a DNS query to 198.51.100.53. The packet arrives on eth1, which is enslaved to mgmt, so the kernel resolves it in table 1001. The leaked route matches and names eth0 as the output device — an interface in the default context. The kernel forwards the packet out eth0.

sequenceDiagram
  autonumber
  participant H as host 10.10.0.10
  participant T1 as table 1001 (mgmt)
  participant T2 as table 254 (default)
  participant DNS as DNS 198.51.100.53

  H->>T1: UDP 10.10.0.10 -> 198.51.100.53:53
  Note over T1: leaked route matches<br/>output device eth0
  T1->>DNS: forwarded out eth0
  DNS->>T2: reply, dst 10.10.0.10
  Note over T2: needs a route to 10.10.0.0/24<br/>this is a separate statement
  T2->>H: forwarded out eth1, if the reverse leak exists

Three things to take from that diagram:

1. The source address is not rewritten. The DNS server sees a query from 10.10.0.0/24 and replies to it. Nothing about the leak makes the query look like it came from the router.
2. The reply is resolved in table 254, because it arrives on eth0. Table 254 has no idea 10.10.0.0/24 exists unless a second, separate leak puts it there.
3. Forwarding is a kernel operation. FRR’s only role is the control plane — deciding which routes exist in which table. Once the route is installed, FRR is not in the path.

The consequence is the trap that Part XVI keeps returning to: a leak is one route in one direction. Leaking 198.51.100.53/32 into mgmt without leaking 10.10.0.0/24 into the default context gives you a query that arrives and a reply that has nowhere to go. The symptom is a timeout, which is exactly what a firewall drop looks like, which is why the first hour of the incident is usually spent in the wrong subsystem.

How it fails

The failure modes an engineer working with leaks must recognise, in the order they are worth checking:

• The leak installs nothing. The vrf value is missing from the next hop, or it names a context where the next hop does not exist. The configuration is clean and ip route show table 1001 has no line. This is first because it is invisible everywhere except the kernel table.
• The leak works in one direction only. Both the request and the timeout are real; the reverse leak was never written. Check the other context’s table before touching the firewall.
• import vrf with an empty source RIB. BGP leaking moves BGP paths. A connected route lives in zebra’s RIB, not BGP’s, until a network statement or a redistribute puts it there. Importing a VRF whose BGP instance holds nothing imports nothing, correctly and silently.
• A leak with no filter. import vrf default with no route map on an internet-facing router imports whatever the transit session is carrying. That is not a leak, it is a merge.
• A default route used as a leak. Leaking 0.0.0.0/0 into a consumer VRF is operationally equivalent to having no VRF at all for outbound traffic. It is occasionally the right answer for an internet VRF; it is never the right answer for shared services.
• MTU mismatch across the leak. The two segments have different MTUs and the route is fine, so large packets vanish while ping succeeds. Part XVI lesson 6 treats this as its own anti-pattern.
• Recursion that does not resolve. A leaked route whose next hop is itself only reachable through another leaked route. FRR either resolves it or holds it inactive; either way the kernel table is the answer.

Rollback

Removing a leak is removing a route, and the direction matters as much when you remove it as when you add it.

configure
# Remove one leaked prefix from the consumer VRF
delete vrf name mgmt protocols static route 198.51.100.53/32

# Remove the matching reverse leak from the default context
delete protocols static route 10.10.0.0/24

commit
save

• Use compare before commit and read the whole diff. Deleting a VRF’s protocols subtree can trip reference checks elsewhere in the same commit.
• Use commit-confirm when the change touches a path your own session depends on.
• Verify in the kernel first — ip route show table 1001 and ip route show table 254 — and in FRR second. The kernel view does not depend on knowing the current FRR output format.
• rollback N inside configure returns to a saved revision when the change is larger than one statement.
• Removing a leak that a service depends on is silent until something fails. A host whose NTP leak disappears keeps working for hours and then starts producing log timestamps nobody can correlate. Plan a window.

Production discipline

Cross-course references

The Linux course’s XIX-Linux-NetFoundations and XXII-Linux-NetTroubleshoot cover the kernel routing-table and l3mdev primitives the VyOS VRF tree wraps. vyos-xv-01-vrf-concept and vyos-xv-02-vrf-config cover the VRF itself; vyos-xv-03-vrf-routing-protocols introduces both leak mechanisms alongside per-VRF OSPF and BGP, and this part takes them further. The BGP parts XXIV-BGPSession and XXV-BGPAdvertise cover the session and origination mechanics that import vrf depends on.

Quiz