Skip to main content
RunBook Academy

← All labs in VyOS

Lab · advanced · ~90 min

Lab: VLAN Trunking, Sub-interfaces, and Bonds

B · Nested virtualisationC · Simulation

Objectives

  • Build a trunk carrying three VLANs between two VyOS 1.5 routers and read the kernel mapping in `ip link show`
  • Prove on the wire, with `tcpdump -e`, which frames carry an 802.1Q tag and which do not
  • Diagnose a VLAN-ID mismatch using the five-step flow from `vyos-viii-05-vlan-troubleshoot` rather than by re-reading the configuration
  • Measure what the VLAN tag costs in usable payload instead of assuming a rule
  • Move an existing trunk onto an active-backup bond with no change to the VLAN addressing, and prove failover with a running ping
  • Read `/proc/net/bonding/bond0` well enough to tell a local misconfiguration from a peer that is not participating

Prerequisites

Objective

By the end of this lab you will have built a three-VLAN trunk between two VyOS 1.5 routers, proved from a capture which frames carry an 802.1Q tag and which do not, broken the trunk with a one-digit VLAN-ID mismatch and found it from evidence rather than by re-reading your own configuration, and then moved the whole trunk onto a bond without changing an IP address. That last part is what matters in production: a bond is supposed to be a transparent substitution for a cable, and this is where you find out whether you believe that.

Architecture

Two VyOS routers, four NICs, and one shared isolated segment. The shared segment is the important design choice: in production a bond terminates on a single switch, so both member links land in the same broadcast domain. One hypervisor bridge with all four member NICs on it models exactly that.

              r1                                    r2
   +---------------------+              +---------------------+
   | eth0  mgmt          |              | eth0  mgmt          |
   | eth1  trunk / bond  |              | eth1  trunk / bond  |
   | eth2  bond member   |              | eth2  bond member   |
   +----+-----------+----+              +----+-----------+----+
        |           |                        |           |
        +-----------+------------+-----------+-----------+
                                 |
                    isolated bridge  "LAB-TRUNK"
                 stands in for a managed switch port-channel

   VLAN 10  data   192.0.2.0/24      r1 .1     r2 .2
   VLAN 20  voice  198.51.100.0/24   r1 .1     r2 .2
   VLAN 30  mgmt   203.0.113.0/24    r1 .1     r2 .2

r2 is a VyOS box standing in for the switch on the far end of the trunk. Where its behaviour differs from a real managed switch — and it does, in two places that matter — the lab says so rather than pretending the substitution is free.

InterfaceRoler1r2
eth0management, untouchedyour bridgeyour bridge
eth1trunk (Tasks 1-5), bond member (Task 6 on)LAB-TRUNKLAB-TRUNK
eth2idle (Tasks 1-5), bond member (Task 6 on)LAB-TRUNKLAB-TRUNK

Requirements

  • A hypervisor with roughly 4 GiB of free RAM and 20 GiB of free disk. Each router needs 1 GiB of RAM and 8 GiB of disk as a floor.
  • The VyOS 1.5 LTS ISO. Every command is written against the 1.5 tree.
  • One isolated layer-2 bridge with no physical port and no host IP address, carrying all four member NICs. On Proxmox that is a vmbr with no bridge-ports line, and the per-NIC Proxmox firewall must be off on every routed interface, per vyos-iv-02-vm-deployment.
  • virtio NICs. e1000 works, but the driver name appears in your evidence.
  • Console access to both routers, per the callout above.

Scenario

The site has one cable between the router and the access switch and three broadcast domains that have to share it: data, voice, and a management VLAN the network team insists on keeping separate. You are going to carry all three over one trunk, and then — because the site is getting a second cable next quarter — rebuild the trunk on a bond so that the migration is a change-window exercise rather than a redesign.

Tasks

Configuration blocks are written for the [edit] prompt. Where one opens with configure and you are already at [edit], skip that line; blocks opening with set or delete continue the session you have open. bash blocks are operational mode or the hypervisor host, as their comments say.

Task 1 — Baseline, and an untagged frame to compare against

Capture the starting state on both routers before you type anything. Cleanup compares against these files.

JOURNAL="$HOME/lab05"
mkdir -p "$JOURNAL"
show configuration commands > "$JOURNAL/pre-lab-config.txt"
ip -brief link show > "$JOURNAL/pre-lab-links.txt"
ip -brief address show > "$JOURNAL/pre-lab-addresses.txt"
cat "$JOURNAL/pre-lab-links.txt"

Now build a plain untagged link so you have something to compare the tagged case against. On r1:

configure
set system host-name r1
set interfaces ethernet eth1 address 192.0.2.1/24
set interfaces ethernet eth1 description 'LAB-TRUNK untagged baseline'
commit

On r2:

configure
set system host-name r2
set interfaces ethernet eth1 address 192.0.2.2/24
set interfaces ethernet eth1 description 'LAB-TRUNK untagged baseline'
commit

Start a capture on r2 and ping from r1:

### On r2
tcpdump -ni eth1 -e -c 4 icmp
### On r1
ping -c 4 192.0.2.2

Look at the capture line. It has an Ethernet header — source MAC, destination MAC, ethertype IPv4 — and no VLAN field at all. Save it. That absence is the control for everything in Task 3.

Then remove the untagged address on both routers, because the same subnet is about to belong to VLAN 10 and having it in two places is confusing rather than instructive:

delete interfaces ethernet eth1 address 192.0.2.1/24
commit

vif N is the whole mechanism. There is no mode trunk directive in VyOS and no top-level vlan tree; creating sub-interfaces is how a port becomes a trunk.

On r1:

configure
set interfaces ethernet eth1 description 'LAB-TRUNK to r2, VLANs 10,20,30'
set interfaces ethernet eth1 vif 10 address 192.0.2.1/24
set interfaces ethernet eth1 vif 10 description 'DATA'
set interfaces ethernet eth1 vif 20 address 198.51.100.1/24
set interfaces ethernet eth1 vif 20 description 'VOICE'
set interfaces ethernet eth1 vif 30 address 203.0.113.1/24
set interfaces ethernet eth1 vif 30 description 'MGMT'
commit
save

On r2, the same three VLANs with .2 addresses:

configure
set interfaces ethernet eth1 description 'LAB-TRUNK to r1, VLANs 10,20,30'
set interfaces ethernet eth1 vif 10 address 192.0.2.2/24
set interfaces ethernet eth1 vif 10 description 'DATA'
set interfaces ethernet eth1 vif 20 address 198.51.100.2/24
set interfaces ethernet eth1 vif 20 description 'VOICE'
set interfaces ethernet eth1 vif 30 address 203.0.113.2/24
set interfaces ethernet eth1 vif 30 description 'MGMT'
commit
save

Read the kernel mapping. VyOS shows you eth1 vif 10; the kernel shows you something with a different name, and every diagnostic tool below the VyOS CLI uses the kernel’s name:

Read-only / Safer1 · kernel view
$ ip link show
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
  link/ether 52:54:00:aa:00:01 brd ff:ff:ff:ff:ff:ff
5: eth1.10@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
  link/ether 52:54:00:aa:00:01 brd ff:ff:ff:ff:ff:ff
6: eth1.20@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
  link/ether 52:54:00:aa:00:01 brd ff:ff:ff:ff:ff:ff
7: eth1.30@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
  link/ether 52:54:00:aa:00:01 brd ff:ff:ff:ff:ff:ff

Illustrative output

Note the MAC address. All four devices share the parent’s MAC — the sub-interfaces are not separate stations on the wire, they are the same station speaking three languages. That single fact explains a large fraction of VLAN confusion: nothing about a sub-interface is separate except the tag and the routing table entry.

Prove each VLAN carries traffic:

ping -c 2 192.0.2.2
ping -c 2 198.51.100.2
ping -c 2 203.0.113.2
show interfaces ethernet eth1 vif 10

All three should answer. If one does not, stop and use the flow in Task 4 before configuring anything else — a VLAN that fails now will fail more confusingly once the bond is underneath it.

Task 3 — Prove the tag on the wire

The -e flag prints the link-layer header, which is where the 802.1Q tag lives. Without it, tcpdump shows you an ordinary IP packet and you have no way to tell which VLAN carried it.

Run a capture on the parent interface on r2 — not on the sub-interface, because the sub-interface sees frames after the tag has been stripped:

### On r2
tcpdump -ni eth1 -e -c 4 vlan 20
### On r1
ping -c 4 198.51.100.2
Read-only / Safer2 · tagged frame on the parent
$ tcpdump -ni eth1 -e -c 1 vlan 20
52:54:00:aa:00:01 > 52:54:00:bb:00:01, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4 (0x0800), 198.51.100.1 > 198.51.100.2: ICMP echo request, id 1234, seq 1, length 64

Illustrative output

Put that line next to the Task 1 capture. Same MACs, same wire, one extra field, and that field is the entire mechanism.

Now prove the separation, which is the claim people make loosest. With the same ping running, capture on the wrong sub-interface:

### On r2
timeout 10 tcpdump -ni eth1.10 icmp

Nothing arrives. VLAN 20 frames are delivered to eth1.20 and nowhere else, because the kernel demultiplexes on the tag before the packet reaches any sub-interface. That is stronger than “they are on different subnets”: they are on different interfaces, and a firewall rule bound to eth1.10 will never see a VLAN 20 packet.

Task 4 — Break it, then find it from evidence

Change one digit on r2 so its VLAN 20 sub-interface becomes VLAN 21:

On r2:

configure
delete interfaces ethernet eth1 vif 20
set interfaces ethernet eth1 vif 21 address 198.51.100.2/24
set interfaces ethernet eth1 vif 21 description 'VOICE'
commit

On r1, ping -c 3 198.51.100.2 now fails. Resist opening r2’s configuration. Work the flow from vyos-viii-05-vlan-troubleshoot in order, and write down what each step rules out.

### Step 1 - is the link up at all?
show interfaces ethernet eth1

### Step 2 - are tagged frames for this VLAN arriving?
tcpdump -ni eth1 -e -c 4 vlan 20

### Step 3 - is the local sub-interface up and addressed?
ip link show eth1.20
ip -brief address show eth1.20

### Step 4 - is there a route?
show ip route 198.51.100.0/24

Run the same Step 2 capture on r2 while r1 pings. Here is the finding that names the fault: on r1 the capture shows echo requests leaving with vlan 20; on r2 the capture on the parent also shows them arriving with vlan 20. The frames cross the wire correctly. They are simply delivered to nothing, because r2 has no eth1.20 for the kernel to demultiplex them into.

That is the signature to memorise. Tagged frames present on the parent and absent on every sub-interface means the VLAN ID exists on the wire and not in the configuration — a trunk mismatch, not a cabling or routing problem. On a real switch the same signature means the port’s allowed-VLAN list does not include the tag.

Confirm and fix:

### On r2
ip link show | grep 'eth1\.'

On r2:

delete interfaces ethernet eth1 vif 21
set interfaces ethernet eth1 vif 20 address 198.51.100.2/24
set interfaces ethernet eth1 vif 20 description 'VOICE'
commit
save

Re-run the ping from r1. It answers.

Task 5 — Measure what the tag costs, do not assume it

The 802.1Q header is four bytes. Whether they come out of your usable payload depends on where you measure and on what the intervening equipment counts as its MTU, and that differs between platforms often enough that the only trustworthy method is measurement on the path you actually have.

From r1, across VLAN 10:

PEER=192.0.2.2
for SIZE in 1464 1468 1472 1473 1476; do
  printf '%s bytes: ' "$SIZE"
  if ping -M do -s "$SIZE" -c 1 -W 2 "$PEER" > /dev/null 2>&1; then
    echo pass
  else
    echo fail
  fi
done

-M do sets the Don’t Fragment bit, so nothing on the path is allowed to split the packet; -s is the ICMP payload size, to which the kernel adds 8 bytes of ICMP header and 20 of IPv4 header. A payload of 1472 therefore makes a 1500-byte IP packet.

Record the largest size that passes. Then run the first failing size again without redirecting the output, and read the message, because there are two completely different failures here and they look identical in a pass/fail loop:

PEER=192.0.2.2
ping -M do -s 1473 -c 1 -W 2 "$PEER"

A local error naming the message size means you exceeded the MTU of your own sub-interface and the packet never left the box. Loss with no error means the packet left and something on the path dropped it — the far end, the bridge, or a device between them. The first is a local configuration fact; the second is a path fact, and confusing them is how MTU tickets end up assigned to the wrong team.

To push further, raise both the parent and the hypervisor bridge above 1500 and repeat. Raising only the router’s MTU measures the bridge, not the router, and produces a confidently wrong answer.

Task 6 — Move the trunk onto a bond

Nothing about the VLAN configuration changes here except the device it hangs from. That is the promise of a bond, and this task is where you check it.

A bond member cannot carry an address or a sub-interface, so the three vif nodes come off eth1 first. (eth1 has held no address of its own since Task 1; if yours still does, delete it in the same commit — VyOS rejects a delete of a node that is not there, so only delete what show configuration commands | match 'eth1' says exists.) Do the whole thing in one commit on each router: the intermediate state, with no trunk anywhere, is not one to leave a box in.

On r1:

configure
delete interfaces ethernet eth1 vif 10
delete interfaces ethernet eth1 vif 20
delete interfaces ethernet eth1 vif 30
set interfaces bonding bond0 mode 'active-backup'
set interfaces bonding bond0 description 'LAB-TRUNK to r2'
set interfaces bonding bond0 member interface eth1
set interfaces bonding bond0 member interface eth2
set interfaces bonding bond0 vif 10 address 192.0.2.1/24
set interfaces bonding bond0 vif 10 description 'DATA'
set interfaces bonding bond0 vif 20 address 198.51.100.1/24
set interfaces bonding bond0 vif 20 description 'VOICE'
set interfaces bonding bond0 vif 30 address 203.0.113.1/24
set interfaces bonding bond0 vif 30 description 'MGMT'
commit
save

On r2, the same with .2 addresses:

configure
delete interfaces ethernet eth1 vif 10
delete interfaces ethernet eth1 vif 20
delete interfaces ethernet eth1 vif 30
set interfaces bonding bond0 mode 'active-backup'
set interfaces bonding bond0 description 'LAB-TRUNK to r1'
set interfaces bonding bond0 member interface eth1
set interfaces bonding bond0 member interface eth2
set interfaces bonding bond0 vif 10 address 192.0.2.2/24
set interfaces bonding bond0 vif 10 description 'DATA'
set interfaces bonding bond0 vif 20 address 198.51.100.2/24
set interfaces bonding bond0 vif 20 description 'VOICE'
set interfaces bonding bond0 vif 30 address 203.0.113.2/24
set interfaces bonding bond0 vif 30 description 'MGMT'
commit
save

The kernel now shows bond0 with bond0.10, bond0.20, bond0.30 hanging from it, and eth1 and eth2 carrying the SLAVE flag:

ip link show bond0
ip link show | grep -E 'bond0|eth1|eth2'
show interfaces bonding

Every ping from Task 2 must still work, unchanged, with the same addresses:

ping -c 2 192.0.2.2
ping -c 2 198.51.100.2
ping -c 2 203.0.113.2

Task 7 — Prove failover, with traffic running

Start a continuous ping on r1 in one session and watch the bond in another, then take the active member down.

### Session 1 on r1 - leave this running
ping -i 0.2 192.0.2.2
### Session 2 on r1 - which member is active right now?
sudo cat /proc/net/bonding/bond0

The file names the mode, the currently active slave, and each member’s MII Status and Link Failure Count. Copy it into the journal before you break anything; a failover with no “before” to compare against is an anecdote.

Now down the active member. If the file reports eth1 as the active slave, disable eth1:

On r1:

configure
set interfaces ethernet eth1 disable
commit
Service impact possibler1 · after the member is downed
$ sudo cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v6.6

Bonding Mode: fault-tolerance (active-backup)
Currently Active Slave: eth2
MII Status: up

Slave Interface: eth1
MII Status: down
Link Failure Count: 1

Slave Interface: eth2
MII Status: up
Link Failure Count: 0

Illustrative output

Go back to the ping and count the lost sequence numbers. That gap is your failover time; record the number, not an adjective.

Restore, and confirm the count does not go back to zero:

On r1:

delete interfaces ethernet eth1 disable
commit
save
sudo cat /proc/net/bonding/bond0

Link Failure Count: 1 persists on eth1. Outside an incident that is the most useful field in the file: a member with a non-zero and growing count is a flapping link, invisible to every reachability test because the bond is doing its job of hiding it. Sample this file from monitoring, not only from an SSH session during an outage.

Task 8 — Try 802.3ad and read the diagnosis

This task is here for what it teaches when it fails. On r1 only, switch the mode:

configure
set interfaces bonding bond0 mode '802.3ad'
set interfaces bonding bond0 lacp-rate 'fast'
commit
sudo cat /proc/net/bonding/bond0

Read the LACP section rather than the reachability. What matters is whether each member reports a non-zero Aggregator ID, and what the partner state says — an aggregation that never forms shows a partner that is not answering, not a local syntax problem. On a lab built over a software bridge with no LAG configured, expect it not to form; on a real switch with a matching port-channel, expect it to.

Record what you saw. Then decide, and write the decision down: keep 802.3ad and fix the far end, or revert to active-backup because the far end will never be a LAG. Both are legitimate; shipping an 802.3ad bond into a peer that cannot aggregate is not.

set interfaces bonding bond0 mode 'active-backup'
delete interfaces bonding bond0 lacp-rate
commit
save

Validation

  • ip link show on both routers lists bond0 plus bond0.10, bond0.20 and bond0.30, and shows eth1 and eth2 with the SLAVE flag.
  • ping succeeds across all three VLAN subnets from r1 to r2, using the same addresses that worked before the bond existed.
  • Your journal holds two tcpdump lines from the same wire: one with no VLAN field, one with ethertype 802.1Q and vlan 20.
  • A capture on eth1.10 while VLAN 20 traffic is flowing shows nothing.
  • The MTU loop has a recorded largest-passing size, and you can state whether the first failing size failed locally or on the path, quoting the message.
  • /proc/net/bonding/bond0 on r1 shows Link Failure Count: 1 on the member you downed, and Currently Active Slave naming the other member at the moment of the failure.
  • You can state, in one sentence each, why active-backup needed nothing from r2 and why 802.3ad did.

Expected Outcome

Both routers end with the trunk on a bond and the VLAN addressing untouched since Task 2:

interfaces {
    bonding bond0 {
        description "LAB-TRUNK to r2"
        member {
            interface eth1
            interface eth2
        }
        mode active-backup
        vif 10 {
            address 192.0.2.1/24
            description "DATA"
        }
        vif 20 {
            address 198.51.100.1/24
            description "VOICE"
        }
        vif 30 {
            address 203.0.113.1/24
            description "MGMT"
        }
    }
    ethernet eth1 {
        description "LAB-TRUNK to r2, VLANs 10,20,30"
    }
    ethernet eth2 {
    }
}

The three VLANs reach across the bond, one member can be removed without dropping more than a fraction of a second of traffic, and you have evidence for every claim in that sentence.

Troubleshooting

A vif commit is rejected because the interface has an address. A bond member cannot hold an address or a sub-interface. Delete both in the commit that creates the bond, as Task 6 does.

One VLAN fails and the others are fine. A per-VLAN fact is a VLAN-ID or sub-interface problem, not a link problem. Run Task 4’s comparison: capture on the parent at both ends and see whether the tag crosses the wire.

Everything fails and ip link show says the sub-interfaces are up. Up means configured, not connected. Check the parent’s LOWER_UP flag, then the bridge on the hypervisor.

/proc/net/bonding/bond0 does not exist. The bond was never created. A bonding node with no member interface commits and produces a device with nothing in it — check show interfaces bonding for a member list first.

Failover works but the ping never recovers. The peer’s forwarding entry for the bond MAC still points at the old port. On a real switch that resolves as soon as the new member transmits; if it does not, the two members are not in the same broadcast domain — on this lab, different bridges.

Cleanup

Cleanup restores both routers to the Task 1 capture, not merely to “no bond”.

Step 1. Find out what is actually there first — VyOS rejects a delete of a node that does not exist, and a cleanup that aborts halfway is worse than none:

show configuration commands | match 'interfaces bonding'
show configuration commands | match 'interfaces ethernet eth1'
show configuration commands | match 'interfaces ethernet eth2'

Then delete exactly what that listed. A reader who finished the lab has the bond and a leftover description on eth1:

configure
delete interfaces bonding bond0
delete interfaces ethernet eth1 description
commit
save

A reader who stopped after Task 5 has the three vif nodes on eth1 instead and no bond, so the deletes are these:

configure
delete interfaces ethernet eth1 vif 10
delete interfaces ethernet eth1 vif 20
delete interfaces ethernet eth1 vif 30
delete interfaces ethernet eth1 description
commit
save

Step 2. Prove it, on each router:

JOURNAL="$HOME/lab05"
show configuration commands > "$JOURNAL/post-lab-config.txt"
ip -brief link show > "$JOURNAL/post-lab-links.txt"
diff -u "$JOURNAL/pre-lab-config.txt" "$JOURNAL/post-lab-config.txt" \
  && echo 'CONFIG RESTORED'
diff -u "$JOURNAL/pre-lab-links.txt" "$JOURNAL/post-lab-links.txt" \
  && echo 'LINKS RESTORED'

A non-empty link diff usually means a bond0 device is still present. Check ip link show bond0 before deleting anything by hand: a device the configuration no longer describes is a commit that did not complete, and is worth reading show log over rather than papering over.

Step 3. If the routers exist only for this lab, destroy them.

### On the Proxmox host, once per router
VMID=201
qm stop "$VMID"
qm destroy "$VMID"

Production notes

Adding a VLAN to an existing trunk is additive on the router and not on the switch, whose allowed-VLAN list is usually owned by another team. Sequence the switch side first: a tag arriving at a router with no matching sub-interface is discarded harmlessly, while a router sending a tag the switch does not allow looks, from the router, exactly like a broken cable.

Removing a VLAN is the dangerous direction — delete interfaces ethernet eth1 vif 20 takes down a subnet’s gateway in one commit with no warning. Use commit-confirm (see vyos-lab-03-commit-rollback), and check show interfaces ethernet eth1 vif 20 for counters that are still moving before deciding the VLAN is unused.

Migrating a live trunk onto a bond — Task 6 — is a genuine outage on a single-homed link, because there is no moment when the addresses exist on both eth1 and bond0. The mitigation is to build the bond with only the second cable as a member, move the addressing to it while the first cable still carries the old trunk, then add the first cable. Two windows and one brief overlap, which is why the second cable gets pulled before the change.

Holding applies here too. A bond running on one member because the other flaps is degraded, not down, and leaving it there with an owner and an end time on the ticket often beats chasing a mode change at 02:00.

What You Learned

  • vif N is the whole trunk mechanism. There is no mode to set, and the kernel names the result eth1.10.
  • Sub-interfaces share the parent’s MAC. One station speaking several languages, not several stations — the tag is all that separates them.
  • Where you capture decides what you see. The tag is on the parent and already gone on the sub-interface.
  • A VLAN-ID mismatch produces no error anywhere. You found it by comparing captures at two ends, which is the only method that works.
  • The tag overhead is measured, not recited. This course states the rule two incompatible ways; ping -M do settles it for your path in seconds.
  • A bond is transparent to the layer above and not to the peer below. The VLANs moved without touching an address, and the mode you may use is decided by whoever owns the far end.

Deliverables

  • · A lab journal holding `ip link show`, `show interfaces ethernet` and `tcpdump -e` evidence for each VLAN, before and after the bond
  • · Two tcpdump captures side by side: one untagged frame and one 802.1Q-tagged frame from the same wire
  • · The measured maximum ICMP payload across a VLAN sub-interface, with the exact failure message for the first size that does not pass
  • · A `/proc/net/bonding/bond0` capture from before, during and after the member failure

Verification status

Last reviewed
2026-08-19
Executed end to end
not yet run on hardware

The commands and configuration here have been reviewed against the verified software versions, but nobody has run this lab start to finish on a system meeting its prerequisites. Treat the Expected Outcome as the intended result rather than an observed one, and keep the Cleanup section to hand.