VRF routing protocols — OSPF, BGP, and crossing the VRF boundary on purpose

A VRF is a routing-table boundary. To make it useful, the engineer runs routing protocols inside it, and — sooner than they expect — needs a specific prefix to cross the boundary anyway: a shared DNS resolver, a monitoring collector, an internet default. Both halves of that are in this lesson, because the second half is where VRF designs fail. A leak that is configured wrongly does not error. It commits, it appears in the configuration, and it installs nothing.

The critical configuration-tree fact for the first half: routing-protocol stanzas must live under vrf name X, not at the top level. A top-level protocols ospf block installs into the main table regardless of which VRF the interface is in.

The placement rule

# WRONG - OSPF runs in the main table even though eth1 is in CUST-A
set protocols ospf area 0 network 10.1.0.0/24
set interfaces ethernet eth1 vrf CUST-A

# RIGHT - OSPF runs inside CUST-A
set vrf name CUST-A protocols ospf area 0 network 10.1.0.0/24
set interfaces ethernet eth1 vrf CUST-A
set interfaces ethernet eth1 address 10.1.0.1/24

The VyOS tree mirrors the FRR tree. Anything under vrf name CUST-A protocols is rendered inside an FRR vrf CUST-A block or as a vrf CUST-A-qualified router instance:

! from the top-level stanza
router ospf
 network 10.1.0.0/24 area 0
!
! from the per-VRF stanza
router ospf vrf CUST-A
 network 10.1.0.0/24 area 0

The vrf CUST-A qualifier is what makes FRR allocate the OSPF instance inside that routing context. Without it the process runs against the main RIB, and the symptom is not a broken adjacency — it is a working adjacency whose routes land in the wrong table.

flowchart TB
  subgraph TOP["Top-level configuration"]
    GOSPF["protocols ospf"] --> GRIB["main RIB"]
    GBGP["protocols bgp"] --> GRIB
  end
  subgraph PERVRF["vrf name CUST-A protocols ..."]
    VOSPF["ospf"] --> VRIB["CUST-A RIB"]
    VBGP["bgp"] --> VRIB
    VSTATIC["static"] --> VRIB
  end
  GRIB --> Z["zebra"]
  VRIB --> Z
  Z --> KT254["kernel table 254"]
  Z --> KT1001["kernel table 1001"]

OSPF inside a VRF

configure
set vrf name CUST-A table 1001
set interfaces ethernet eth1 vrf CUST-A
set interfaces ethernet eth1 address 10.1.0.1/24

# OSPF inside the VRF
set vrf name CUST-A protocols ospf parameters router-id 10.1.0.1
set vrf name CUST-A protocols ospf area 0 network 10.1.0.0/24

# Turn on authentication for the area, then supply the key per interface
set vrf name CUST-A protocols ospf area 0 authentication plaintext-password
set vrf name CUST-A protocols ospf interface eth1 authentication plaintext-password s3cr3t

# Advertise a default into the VRF, but only if we have one to give
set vrf name CUST-A protocols ospf default-information originate metric 10 metric-type 2

commit
save

Three details in that block are the ones people get wrong.

The router id lives under parameters. It is protocols ospf parameters router-id, not protocols ospf router-id. Set it explicitly per VRF. If you do not, FRR picks one from the addresses it can see in that routing context, which means the id can change when an interface is added or removed — and an OSPF instance that changes router id resets its adjacencies.

Area authentication and the key are two separate commands. area 0 authentication plaintext-password selects the authentication type for the area and takes no key. The key is a per-interface value under protocols ospf interface eth1 authentication plaintext-password. A plain text key is capped at 8 characters and an MD5 key at 16, so a password policy that mandates longer will simply not commit; use interface eth1 authentication md5 key-id 1 md5-key ... when you need the stronger option.

default-information originate is how a default enters OSPF. Without always, FRR only originates it when the VRF actually has a default route of its own. Adding always makes the router advertise a default it may not be able to honour, which turns every other router in the area into a black-hole client if the upstream is down.

Validation is the ordinary FRR command set with a vrf qualifier on every one:

show ip ospf vrf CUST-A
show ip ospf vrf CUST-A neighbor
show ip ospf vrf CUST-A route
show ip route vrf CUST-A

Read show ip ospf vrf CUST-A neighbor for two things: a neighbour in Full (or Full/DR / Full/BDR on a broadcast segment), and the interface column naming the interface you expect. A neighbour stuck in Init means hellos are arriving but yours are not getting back — usually authentication mismatch, mismatched hello/dead timers, or a unidirectional path. A neighbour stuck in ExStart means the adjacency negotiated and the database exchange failed, which on a VRF is very often an MTU mismatch between the VRF-facing interface and the peer.

BGP inside a VRF

configure
set vrf name CUST-A table 1001
set interfaces ethernet eth1 vrf CUST-A
set interfaces ethernet eth1 address 10.1.0.1/24

# eBGP to the tenant's edge router, inside the VRF
set vrf name CUST-A protocols bgp system-as 65000
set vrf name CUST-A protocols bgp parameters router-id 10.1.0.1
set vrf name CUST-A protocols bgp neighbor 10.1.0.254 remote-as 65001
set vrf name CUST-A protocols bgp neighbor 10.1.0.254 address-family ipv4-unicast
set vrf name CUST-A protocols bgp neighbor 10.1.0.254 address-family ipv4-unicast soft-reconfiguration inbound

# Originate the tenant LAN into the session
set vrf name CUST-A protocols bgp address-family ipv4-unicast network 10.1.0.0/24

commit
save

The ASN is system-as in VyOS 1.4 and 1.5. The older set protocols bgp 65000 ... form, where the ASN was a node name rather than a value, was removed in 1.4; a configuration written that way will not commit on 1.5. system-as is mandatory — a BGP stanza without one fails verification with BGP system-as number must be defined!.

Each VRF gets its own BGP instance, and each instance needs its own system-as even when the number is identical to the global one. Set a per-VRF parameters router-id for the same reason as OSPF: the automatic choice is derived from what is visible in that routing context and can move.

show ip bgp vrf CUST-A summary
show ip bgp vrf CUST-A
show ip route vrf CUST-A

In the summary, the column to read is the state column. It shows a state name (Idle, Connect, Active, OpenSent, OpenConfirm) while the session is down and a prefix count once it is Established — a number in that column means the session is up and that many prefixes were accepted. Active is the one that misleads people: it means the local side is trying to open a TCP connection and failing, not that anything is active.

What does not belong inside the VRF

When the VyOS box is a genuine L3VPN PE, the per-VRF address family carries the RD, the route targets, and the instruction to exchange with the VPN RIB:

set vrf name CUST-A protocols bgp system-as 65000
set vrf name CUST-A protocols bgp parameters router-id 10.0.0.1
set vrf name CUST-A protocols bgp address-family ipv4-unicast rd vpn export '65000:1001'
set vrf name CUST-A protocols bgp address-family ipv4-unicast route-target vpn both '65000:1001'
set vrf name CUST-A protocols bgp address-family ipv4-unicast export vpn
set vrf name CUST-A protocols bgp address-family ipv4-unicast import vpn

The PE-to-PE session itself does not. It lives in the default instance, carries the ipv4-vpn family, and is what actually moves the VPN routes between routers:

set protocols bgp system-as 65000
set protocols bgp parameters router-id 10.0.0.1
set protocols bgp neighbor 10.0.0.2 remote-as 65000
set protocols bgp neighbor 10.0.0.2 update-source lo
set protocols bgp neighbor 10.0.0.2 address-family ipv4-vpn

That split is the part worth memorising. rd, route-target, export vpn and import vpn describe how one VRF’s routes are stamped on the way into the VPN RIB and selected on the way out. The ipv4-vpn neighbour describes how that RIB reaches the other PE. Putting the VPNv4 neighbour inside the VRF does not build an L3VPN; it builds an ordinary session in the tenant’s routing context.

Crossing the VRF boundary on purpose

Every multi-tenant design eventually needs one prefix to cross. VyOS gives you two mechanisms, and it is worth being precise about them, because the command that looks like it should work does not.

Why the obvious command does nothing

Take a shared services subnet 10.99.0.0/24, reachable from the default VRF via 192.0.2.1 on eth0. The intuitive way to give CUST-A access:

set vrf name CUST-A protocols static route 10.99.0.0/24 next-hop 192.0.2.1

This commits. It appears in show configuration commands. It renders to FRR as:

vrf CUST-A
 ip route 10.99.0.0/24 192.0.2.1
exit-vrf

And it does nothing at all, because next hops are resolved in the routing context the route belongs to. FRR looks for a way to reach 192.0.2.1 inside CUST-A. Table 1001 contains the tenant’s connected route and nothing else. The next hop does not resolve, the route stays inactive, and zebra never offers it to the kernel.

$ ip route show table 1001

10.1.0.0/24 dev eth1 proto kernel scope link src 10.1.0.1

FRR will show the route with an unresolved next hop and no selection marker. The exact wording differs between FRR releases, so treat that as a hint and the kernel table as the fact:

$ vtysh -c 'show ip route vrf CUST-A 10.99.0.0/24'

VRF CUST-A:
S   10.99.0.0/24 [1/0] via 192.0.2.1 inactive

Two things distinguish this from a working route: there is no * (selected) or > (installed in the FIB) in the code column, and the next hop is flagged as not resolving. A working static in the same table would read S>* 10.99.0.0/24 [1/0] via 192.0.2.1, eth0.

This is the failure this part of the course exists to teach. Nothing errors. The change review passes. The tenant reports that the shared service is unreachable, and the configuration says it should not be.

Mechanism one: a static route whose next hop lives in another VRF

The static-route tree has a vrf node under both next-hop and interface. Its help text is exactly what it does: VRF to leak route.

# From CUST-A, reach the shared subnet via a next hop in the default VRF
set vrf name CUST-A protocols static route 10.99.0.0/24 next-hop 192.0.2.1 vrf default

The value is either default — meaning the main table — or the name of another configured VRF. This renders to FRR as:

vrf CUST-A
 ip route 10.99.0.0/24 192.0.2.1 nexthop-vrf default
exit-vrf

nexthop-vrf tells FRR to resolve 192.0.2.1 in the default routing context while installing the resulting route in CUST-A’s. Now it resolves — 192.0.2.0/24 is connected on eth0 in the main table — and zebra installs it:

$ ip route show table 1001

10.1.0.0/24 dev eth1 proto kernel scope link src 10.1.0.1
10.99.0.0/24 via 192.0.2.1 dev eth0 proto static

The signature of a leaked route in a kernel table is exactly that: an output device that is not a member of the VRF the table belongs to. eth0 is in the default VRF; the route is in table 1001. If you can see that line, the leak is real.

The return path is a separate configuration

A leak is unidirectional. 10.99.0.0/24 is now reachable from CUST-A, but the shared service’s replies are routed by the main table, which has no route to 10.1.0.0/24. Traffic goes out and nothing comes back — which reads exactly like a firewall drop and sends people to the wrong subsystem for an hour.

The reverse leak is configured in the default context, pointing into the VRF:

set protocols static route 10.1.0.0/24 interface eth1 vrf CUST-A

Rendering to:

ip route 10.1.0.0/24 eth1 nexthop-vrf CUST-A

Here the interface form is the right one, because eth1 is directly connected inside CUST-A and there is no next-hop address to name. Verify the same way, in the main table this time:

ip route show table 254

sequenceDiagram
  autonumber
  participant H as Host 10.1.0.50 (CUST-A)
  participant K1 as table 1001
  participant K2 as table 254
  participant S as Shared service 10.99.0.5

  H->>K1: dst 10.99.0.5
  Note over K1: leaked route<br/>via 192.0.2.1 dev eth0
  K1->>S: forwarded out eth0
  S->>K2: reply, dst 10.1.0.50
  Note over K2: without the reverse leak<br/>no route to 10.1.0.0/24
  K2-->>S: unreachable

Mechanism two: BGP import vrf

Static leaks are one prefix at a time and they do not follow topology changes. When there are many prefixes, or when the set of prefixes is learned rather than known, BGP does the same job dynamically.

# Both instances need an ASN; the number may be the same
set protocols bgp system-as 65000
set protocols bgp parameters router-id 10.0.0.1
set vrf name CUST-A protocols bgp system-as 65000
set vrf name CUST-A protocols bgp parameters router-id 10.1.0.1

# Put the tenant's prefixes into CUST-A's BGP RIB in the first place
set vrf name CUST-A protocols bgp address-family ipv4-unicast redistribute connected

# Import in each direction
set vrf name CUST-A protocols bgp address-family ipv4-unicast import vrf default
set protocols bgp address-family ipv4-unicast import vrf CUST-A

The redistribute connected line is not optional decoration, and leaving it out is the second-most-common way to configure a leak that does nothing. import vrf copies routes between BGP RIBs. A connected route in CUST-A is in zebra’s RIB, not in BGP’s, until something puts it there — a network statement or a redistribute. Import a VRF whose BGP instance has no routes and you import nothing, correctly and silently.

Filter what crosses with a route map. The map is applied to the address-family that is doing the importing:

set policy prefix-list SHARED-ONLY rule 10 action permit
set policy prefix-list SHARED-ONLY rule 10 prefix 10.99.0.0/24

set policy route-map LEAK-IN rule 10 action permit
set policy route-map LEAK-IN rule 10 match ip address prefix-list SHARED-ONLY

set vrf name CUST-A protocols bgp address-family ipv4-unicast route-map vrf import LEAK-IN

Without the map, import vrf default imports the whole main table’s BGP routes into the tenant — including, on an internet-facing router, whatever the transit session is carrying. That is not a leak, it is a merge.

What VyOS refuses

The VPN mechanism and the import vrf mechanism are two implementations of the same idea, and they cannot both be configured on one address family. VyOS rejects the combination outright, with messages that name the pair: Command "import vrf" conflicts with "rd vpn export" command!, and the same form for route-target vpn both, import and export.

Two more references block a delete:

• A VRF that another VRF imports from cannot be removed: Cannot delete VRF instance "CUST-A", unconfigure "import vrf" commands!
• The default BGP instance cannot be removed while any VRF has a BGP instance depending on it.

Both are the validator protecting you from a half-built import relationship, and both are cleared by deleting the dependent side in the same commit.

Common configuration mistakes

1. Protocol stanza at the top level. set protocols ospf area 0 network 10.1.0.0/24 outside vrf name CUST-A. The adjacency comes up, the routes install in the main table, and connectivity to directly connected hosts is unaffected — so nothing looks wrong until someone checks which table the learned prefixes are in.
2. 1.3-era BGP syntax. set protocols bgp 65000 neighbor ... with the ASN as a node name. Removed in 1.4. The 1.5 form is system-as.
3. A leak with no vrf on the next hop. Commits, installs nothing. This is the one to internalise, because it is invisible in the configuration and only visible in the kernel table.
4. A leak in one direction. Traffic leaves, replies have no route home. The symptom mimics a firewall drop.
5. import vrf with nothing in the source BGP RIB. No network statement and no redistribute in the source VRF means there is nothing to import.
6. import vrf with no route map. Imports everything the source instance carries.

How it fails

• The route is inactive. Configured, committed, absent from ip route show table TABLEID. Cause: the next hop does not resolve in the context the route lives in. Fix: add the vrf value to the next hop, or give the VRF a path to the next hop.
• The route installs and traffic still fails. Cause: no reverse leak, or a firewall rule that only ever saw traffic in one routing context. Check the return direction’s table before touching the firewall.
• The adjacency is up in the wrong instance. show ip ospf neighbor shows it, show ip ospf vrf CUST-A neighbor does not. The stanza is at the top level.
• BGP is Established and no prefixes arrive. The neighbour has no address-family configured, so the session negotiated no capability to carry anything.
• The commit is rejected naming two commands. import vrf and the VPN commands were both configured on one address family. Pick one.
• The VRF cannot be deleted. Another VRF imports from it, or its interfaces and static routes are still present. Remove the reference in the same commit.

Rollback

# Remove a routing protocol from a VRF
configure
delete vrf name CUST-A protocols ospf
commit
save

# Remove a leak, both directions
configure
delete vrf name CUST-A protocols static route 10.99.0.0/24
delete protocols static route 10.1.0.0/24
commit
save

# Move a misplaced protocol stanza into its VRF, in one commit
configure
delete protocols ospf
set vrf name CUST-A protocols ospf parameters router-id 10.1.0.1
set vrf name CUST-A protocols ospf area 0 network 10.1.0.0/24
commit
save

A safe rollback sequence:

1. Capture the running configuration before you start.
2. compare and read the whole diff, not just the line you meant to change — deleting a VRF’s protocols subtree can trip the delete-reference checks elsewhere.
3. Use commit-confirm when the change touches a path your session depends on.
4. After commit, verify in the kernel first (ip route show table 1001, ip route show table 254) and in FRR second. The kernel view does not depend on knowing the current FRR output format.
5. save only once both directions of traffic have been re-tested.

Production discipline

Cross-course references

• vyos-xv-01-vrf-concept covers the l3mdev model and the routing-policy rules that make a VRF miss fail rather than fall through.
• vyos-xv-02-vrf-config covers the VRF itself: names, table ids, and the reference checks that block a delete.
• vyos-xvi-01-leaking-concept and the rest of Part XVI take route leaking further, including the firewall and IPv6 dimensions and the anti-patterns.
• The BGP course’s XXIV-BGPSession, XXV-BGPAdvertise and XXXI-BGPTroubleshoot cover the session and policy mechanics that this lesson applies inside a VRF.
• The OSPF course’s XVIII-OSPFFundamentals through XXII-OSPFTroubleshoot do the same for OSPF.

Quiz