VXLAN with BGP EVPN — EVPN Type-2/3/5 routes, BGP as control plane

BGP EVPN (RFC 7432, adapted to VXLAN by RFC 8365) is the modern control plane for VXLAN. It replaces the flood-and-learn mechanism (which uses multicast to propagate MAC learning) with a BGP-based mechanism: VTEPs advertise their MAC/IP/VNI information via BGP EVPN routes. Other VTEPs learn the remote MACs from BGP, eliminating the need for multicast in the underlay and providing a scalable, deterministic control plane.

This lesson covers BGP EVPN as the VXLAN control plane, the EVPN route types (Type-2, Type-3, Type-5), and the VyOS 1.5 configuration surface that implements them.

Why BGP EVPN

Traditional VXLAN uses flood-and-learn: a VTEP that does not know the destination MAC floods the packet to all VTEPs in the same VNI (via multicast, or via a static remote list). Remote VTEPs learn the source MAC from the frames they receive. This works but requires multicast in the underlay or hand-maintained VTEP lists, is inefficient, and does not scale.

BGP EVPN replaces this with a BGP-based mechanism:

1. VTEPs advertise their MACs via BGP. Each VTEP learns locally-attached MACs from its bridge and advertises them as BGP EVPN Type-2 routes.
2. VTEPs advertise which VNIs they serve. Each VTEP advertises its VTEP IP and its VNI membership as BGP EVPN Type-3 (IMET) routes, which is how the fabric learns where to replicate BUM traffic.
3. Other VTEPs learn from BGP. When a VTEP needs to send a frame to a MAC, it looks up the MAC in its forwarding database, finds the remote VTEP IP that BGP installed, encapsulates and sends.

flowchart LR
  subgraph DC1["Data Center 1"]
    VTEP1["VTEP 1<br/>has VM_A1 (MAC A)"]
    VTEP2["VTEP 2"]
  end
  subgraph DC2["Data Center 2"]
    VTEP3["VTEP 3<br/>has VM_A2 (MAC B)"]
  end
  VTEP1 -- "BGP EVPN Type-2<br/>MAC A, VNI 10001" --> VTEP2
  VTEP2 -- "BGP EVPN Type-2<br/>MAC A" --> VTEP3
  VTEP3 -- "BGP EVPN Type-2<br/>MAC B, VNI 10001" --> VTEP2
  VTEP2 -- "BGP EVPN Type-2<br/>MAC B" --> VTEP1

The control plane (BGP EVPN) carries MAC information; the data plane (VXLAN) carries the actual frames.

EVPN route types

EVPN uses several BGP route types (defined in RFC 7432 and extensions):

Type-2: MAC/IP advertisement

Advertises a MAC address (and optionally an IP address) and the VTEP that owns it. The route carries:

• RD: Route Distinguisher, which makes the route unique per VTEP per VNI.
• Ethernet Segment Identifier (ESI): identifies a multi-homed Ethernet segment. On a single-homed port it is all zeros; it is only non-zero when EVPN multihoming (ESI-LAG) is in use.
• Ethernet Tag ID: zero for the ordinary VLAN-based VXLAN service. The VNI that identifies the L2 broadcast domain travels in the route’s VNI/label field, not here — RFC 8365 is the document that maps EVPN onto VXLAN this way.
• MAC address: the MAC being advertised.
• IP address (optional): the IP associated with the MAC, which is what lets a remote VTEP answer ARP locally instead of flooding.
• Next-hop (VTEP IP): the VTEP that owns this MAC.

The receiving VTEP installs the MAC in its forwarding database with the next-hop being the VTEP IP. When a frame for this MAC arrives, the VTEP encapsulates it in VXLAN with destination = remote VTEP IP.

Type-3: Inclusive Multicast Ethernet Tag (IMET)

Advertises the VTEP’s membership of a VNI, for BUM (Broadcast, Unknown unicast, Multicast) traffic. The route carries:

• RD: Route Distinguisher.
• Ethernet Tag ID: zero, for the same reason as Type-2.
• Originating Router’s IP: the VTEP IP. This is the address other VTEPs will replicate BUM traffic to.
• PMSI Tunnel Attribute: names the replication method and carries the VNI. For VXLAN with FRR this is ingress (head-end) replication.

This route is what builds the flood list. When a VTEP receives a broadcast, unknown-unicast or multicast frame on a VNI, it makes one copy per remote VTEP that advertised an IMET route for that VNI and unicasts each copy. No underlay multicast is involved.

Type-5: IP Prefix Route (IP-VRF)

Advertises an IP prefix rather than a MAC, so that traffic can be routed between subnets across the fabric. The route carries:

• RD: Route Distinguisher.
• IP prefix: the prefix being advertised.
• L3VNI: carried in the route’s VNI/label field; it identifies the tenant IP-VRF the prefix belongs to. The Ethernet Tag ID is zero.
• Router’s MAC extended community and next-hop: the VTEP that can reach the prefix, and the inner destination MAC to put in the VXLAN payload so the far end routes rather than bridges it.

Type-5 is what makes inter-subnet (routed) traffic work in the overlay. It is not on by default: FRR does not export a VRF’s IPv4/IPv6 prefixes into EVPN until you ask it to. The VyOS command for that is shown in the symmetric IRB section below.

What VyOS 1.5 actually exposes

Before configuring anything, it is worth being precise about the surface, because EVPN on VyOS is thinner than the vendor documentation of other platforms suggests.

Everything EVPN-related lives under set protocols bgp address-family l2vpn-evpn (or the same path inside set vrf name ... protocols bgp for a tenant IP-VRF). VyOS renders it into FRR’s address-family l2vpn evpn block:

And the peer side:

Configure BGP EVPN VXLAN on VyOS

The topology continues from Part XLIV-03: two VTEPs, loopbacks 192.0.2.10 and 192.0.2.20, an underlay on 10.0.0.0/24, and one L2VNI (10001) carrying a tenant VLAN off eth1.

VTEP1:

configure

# --- Underlay ---
# The loopback address is the VTEP address. Every EVPN route this
# router originates will carry it as the next-hop.
set interfaces loopback lo address 192.0.2.10/32
set interfaces ethernet eth0 address 10.0.0.1/24

# VXLAN adds 50 bytes of overhead on IPv4. The overlay wants a normal
# 1500-byte MTU, so the underlay must carry at least 1550.
set interfaces ethernet eth0 mtu 9000

# Reachability to the peer VTEP loopback. Part XLIV-03 covers the
# production forms (OSPF, or eBGP between leaf and spine); a static
# route keeps this lesson on the overlay.
set protocols static route 192.0.2.20/32 next-hop 10.0.0.2

# --- Overlay control plane ---
set protocols bgp system-as 65000
set protocols bgp parameters router-id 192.0.2.10
set protocols bgp neighbor 192.0.2.20 remote-as 65000
set protocols bgp neighbor 192.0.2.20 update-source lo

# This line is what activates the peer in the EVPN address family.
set protocols bgp neighbor 192.0.2.20 address-family l2vpn-evpn

# This line is what turns EVPN on at all.
set protocols bgp address-family l2vpn-evpn advertise-all-vni

# --- Overlay data plane ---
# One VXLAN device per L2VNI. No "remote" and no "group": EVPN
# discovers the far-end VTEPs, so a static peer list would fight it.
set interfaces vxlan vxlan10001 vni 10001
set interfaces vxlan vxlan10001 source-address 192.0.2.10
set interfaces vxlan vxlan10001 parameters nolearning

# The VXLAN device is a bridge port, alongside the customer port.
set interfaces bridge br10001 member interface vxlan10001
set interfaces bridge br10001 member interface eth1

commit
save

VTEP2 is the mirror image: lo is 192.0.2.20/32, eth0 is 10.0.0.2/24, the static route points at 192.0.2.10 next-hop 10.0.0.1, router-id and source-address are 192.0.2.20, and the BGP neighbor is 192.0.2.10. The VNI, the bridge name and the AS are identical on both sides.

Why each piece is there

1. update-source lo sources the BGP session from the loopback. The EVPN next-hop is taken from the VXLAN interface’s source-address, not from the session address, but peering on loopbacks keeps the two consistent and survives a link failure in a multi-path underlay.
2. neighbor ... address-family l2vpn-evpn is not optional bookkeeping. VyOS always emits no bgp default ipv4-unicast into FRR, so a neighbor is activated only in the address families you name. Omit this line and the session still reaches Established while carrying no EVPN routes at all.
3. advertise-all-vni is the master switch. It is what makes bgpd ask zebra for the kernel’s VNI list and start originating Type-2 and Type-3 routes.
4. parameters nolearning disables kernel data-plane MAC learning on the VXLAN device. With EVPN, MAC learning is BGP’s job; leaving kernel learning on gives you two sources of truth for the same forwarding database.
5. The bridge is where the L2 domain lives. The VXLAN device and the customer-facing port are both ports on it.

Validation

Work outwards: session, then routes, then the forwarding state the routes produced.

# 1. Is the peer up in the EVPN address family?
show bgp l2vpn evpn summary

# 2. Did EVPN turn on at all? This lists the VNIs zebra found
#    and how many MACs and remote VTEPs each has.
show evpn vni
show evpn vni 10001

# 3. What is in the EVPN table?
show bgp l2vpn evpn

# 4. The MAC/IP (Type-2) routes specifically.
show bgp l2vpn evpn route type macip

# 5. The IMET (Type-3) routes - the BUM flood list.
show bgp l2vpn evpn route type multicast

# 6. Which route-targets is this router importing?
show bgp l2vpn evpn import-rt

# 7. Did the routes become forwarding state?
show evpn mac vni 10001
show evpn next-hops vni all

# 8. The kernel view of the same thing. This is the Linux "bridge"
#    utility run from the VyOS shell, not a VyOS op-mode command.
bridge fdb show dev vxlan10001

$ show evpn vni

VNI        Type VxLAN IF              # MACs   # ARPs   # Remote VTEPs  Tenant VRF
10001      L2   vxlan10001            3        2        1               default

Read that line right to left. # Remote VTEPs is the count of far-end VTEPs learned from Type-3 routes; if it is 0, the control plane is not working regardless of what the BGP table shows. # MACs counts both local and remote MACs for the VNI. Tenant VRF: default means this is a pure L2VNI with no IP-VRF attached.

A clean validation: the peer is Established in show bgp l2vpn evpn summary; show evpn vni lists VNI 10001 with a non-zero remote VTEP count; show bgp l2vpn evpn route type macip shows the remote VTEP’s MACs with the remote loopback as next-hop; show evpn mac vni 10001 shows those MACs pointing at the remote VTEP; and a ping between two hosts on the tenant VLAN succeeds.

Production failure modes

EVPN is configured but nothing is advertised

The session is Established, the VXLAN interface is up, the bridge has ports — and show bgp l2vpn evpn is empty on both sides.

Cause, in order of likelihood: advertise-all-vni is missing. It is the single line that turns the feature on, and every other piece of the configuration looks complete without it.

Diagnostic: show evpn vni lists nothing, because bgpd never asked zebra for the VNI table. From configuration mode, show protocols bgp address-family l2vpn-evpn shows the subtree you actually have.

Fix: set protocols bgp address-family l2vpn-evpn advertise-all-vni.

The peer is Established but carries no EVPN routes

show bgp summary shows the neighbor Established. show bgp l2vpn evpn summary does not list it at all — that command only shows peers that were activated in the EVPN address family.

Cause: the neighbor was never activated in the EVPN address family. Because VyOS emits no bgp default ipv4-unicast, a neighbor participates only in the address families configured under it.

Fix: set protocols bgp neighbor 192.0.2.20 address-family l2vpn-evpn, then commit. The session resets to renegotiate capabilities, which is a brief overlay outage — schedule it.

Route-target mismatch after someone configured one

This is the failure that the auto-derivation was protecting you from. With defaults, the import RT is the wildcard *:VNI and mismatches are impossible. The moment one VTEP is given an explicit route-target import, the wildcard is gone and only literal matches are imported.

Diagnostic: the route is visible in show bgp l2vpn evpn (BGP received it) but absent from show evpn mac vni 10001 (it was never imported into the VNI). show bgp l2vpn evpn import-rt lists the route-targets this router will accept; compare that against the extended community on the route in show bgp l2vpn evpn route detail.

Fix: make the far side’s export RT match this side’s import RT — or, better, delete both explicit route-targets and return to auto-derivation:

delete protocols bgp address-family l2vpn-evpn vni 10001 route-target
commit

Routes arrive but the next-hop is unreachable

show bgp l2vpn evpn shows the remote routes, but they are not marked valid and best, and no traffic flows.

Cause: EVPN routes carry the originating VTEP’s address as next-hop, and BGP will not select a route whose next-hop it cannot resolve. If the underlay does not have a route to 192.0.2.20, the EVPN routes from that VTEP are inert.

Diagnostic: show ip route 192.0.2.20/32. If it is missing, the problem is the underlay, not EVPN.

A related and nastier variant: someone applied nexthop-self on a route reflector. An RR is not a VTEP; rewriting the next-hop to the RR’s own address makes every VTEP encapsulate towards the route reflector, which has no VXLAN device and drops the traffic. Apply nexthop-self only where the router originating the advertisement is also the VTEP.

MTU: the overlay works for pings and fails for real traffic

Small frames pass, large transfers stall. This is the classic VXLAN MTU symptom and it is worth stating precisely: the outer VXLAN encapsulation costs 50 bytes over IPv4 and 70 over IPv6. Those are the exact figures VyOS itself uses when it checks the underlay.

VyOS warns rather than blocks. Setting the VXLAN MTU below 1500 produces RFC7348 recommends VXLAN tunnels preserve a 1500 byte MTU at commit, and using source-interface whose MTU is too small produces Underlying device MTU is too small (... bytes) for VXLAN overhead (50 bytes!). Both are warnings; the commit succeeds and the fabric ships broken.

Fix: raise the underlay to at least 1550 (9000 if the hardware allows) and leave the VXLAN interface at the default 1500. Shrinking the overlay MTU instead pushes the problem onto every tenant workload.

First packets are flooded

A workload sends traffic before its MAC has been advertised. The ingress VTEP has no Type-2 route for the destination, so the frame is BUM: it is replicated to every VTEP that advertised an IMET route for the VNI.

This is not a fault. It is head-end replication doing its job, and it stops as soon as the destination’s Type-2 route arrives. flooding disable turns it off, which is a deliberate choice for a fabric where every MAC is known in advance and unknown-unicast should be dropped rather than replicated — not a fix for a transient.

Type-5 routes are not appearing

Inter-subnet traffic across the fabric fails while intra-subnet traffic works.

Cause: an L2VNI alone never produces Type-5 routes. Type-5 requires an IP-VRF bound to an L3VNI, and requires you to explicitly export the VRF’s prefixes into EVPN.

Symmetric IRB: the L3VNI and Type-5 routes

Routing between subnets in the overlay needs a second kind of VNI — an L3VNI — and a VRF to hold the tenant’s routing table. On VyOS 1.5 the pieces are:

configure

# The tenant IP-VRF, bound to its L3VNI.
set vrf name TENANT-A table 100
set vrf name TENANT-A vni 5000

# The L2VNI's bridge becomes the tenant SVI: it moves into the VRF
# and takes the anycast gateway address.
set interfaces bridge br10001 vrf TENANT-A
set interfaces bridge br10001 address 10.0.1.1/24

# The L3VNI needs its own VXLAN device and bridge, also in the VRF.
set interfaces vxlan vxlan5000 vni 5000
set interfaces vxlan vxlan5000 source-address 192.0.2.10
set interfaces vxlan vxlan5000 parameters nolearning
set interfaces bridge br5000 member interface vxlan5000
set interfaces bridge br5000 vrf TENANT-A

# A BGP instance for the VRF, and the line that exports its prefixes
# into EVPN as Type-5 routes.
set vrf name TENANT-A protocols bgp system-as 65000
set vrf name TENANT-A protocols bgp address-family ipv4-unicast redistribute connected
set vrf name TENANT-A protocols bgp address-family l2vpn-evpn advertise ipv4 unicast

# Advertise the tenant gateway so remote VTEPs can ARP for it locally.
set protocols bgp address-family l2vpn-evpn vni 10001 advertise-default-gw
set protocols bgp address-family l2vpn-evpn vni 10001 advertise-svi-ip

commit
save

advertise ipv4 unicast is the specific thing people miss. FRR does not export a VRF’s IPv4 or IPv6 prefixes to EVPN as Type-5 routes until that command is present in the BGP instance of that VRF — the VRF can be perfectly configured, the L3VNI can be up, and the fabric still carries no prefixes.

Validation for the L3 side:

# Does the VRF have its L3VNI?
show evpn vni 5000

# The router MACs the fabric uses for symmetric routing.
show evpn rmac vni all

# The Type-5 prefix routes.
show bgp l2vpn evpn route type prefix

# The tenant's routing table.
show ip route vrf TENANT-A

show evpn vni 5000 should report the VNI as type L3 with Tenant VRF: TENANT-A. If it reports L2, the VRF-to-VNI binding did not take and no Type-5 route will ever be built.

Rollback

configure

# Save the running configuration before touching anything.
save /config/pre-evpn-rollback.conf

# See exactly what is about to change.
compare

# Remove the overlay control plane, leaving the underlay alone.
delete protocols bgp address-family l2vpn-evpn
delete protocols bgp neighbor 192.0.2.20 address-family l2vpn-evpn

# Remove the overlay data plane.
delete interfaces bridge br10001 member interface vxlan10001
delete interfaces vxlan vxlan10001

# For a change you are not certain of, make it self-reverting.
commit-confirm 5
# ... verify the overlay is still healthy, then accept the change:
confirm

# If verification fails, put the saved file back. This is the soft
# path: it goes through commit like any other change.
load /config/pre-evpn-rollback.conf
commit

Deleting the EVPN address family withdraws every route this VTEP originated; remote VTEPs age out its MACs and its IMET route, and it drops off their flood lists. Deleting the VXLAN interface removes the data plane with it. Note the ordering: taking the control plane down first means remote VTEPs stop sending to you before you stop being able to receive, which is the less disruptive order.

Production discipline

Cross-course references

• Part XLIV-01 (XLIV-VyOS-VXLAN / concept) covers the VXLAN protocol.
• Part XLIV-03 (XLIV-VyOS-VXLAN / underlay) covers the underlay requirements the EVPN next-hops depend on.
• Part XLIV-05 (XLIV-VyOS-VXLAN / without EVPN) covers flood-and-learn VXLAN for comparison.
• Part XLIV-06 (XLIV-VyOS-VXLAN / troubleshoot) covers the diagnostic order in more depth.
• Part XXIX-02 (XXIX-VyOS-BGP / community config) covers BGP communities; EVPN route-targets are extended communities and are matched the same way.

Quiz