Skip to main content
RunBook Academy

VyOSLI · MTU and FragmentationMTU

Tunnel overhead — WireGuard 32-80, IPsec 50-66, GRE 24, VXLAN 50

Advanced⏱ ~26 minshow interfacesip link show <tunnel>ping -M do -s <size>tracepathconfigurecommit-confirmrollback

What you'll learn

  • Calculate the effective MTU for WireGuard, IPsec, GRE, and VXLAN tunnels
  • Configure the inner MTU on tunnel interfaces to account for overhead
  • Recognise the production failure modes (oversized inner packet, jumbo with tunnel overhead, asymmetric tunnel MTU)
  • Validate the tunnel MTU end-to-end with ping -M do -s and tracepath
  • Roll back tunnel MTU changes safely with commit-confirm

Prerequisites

Verified against VyOS 1.5.x LTS (circinus) · VyOS 1.4.x (sagitta) — legacy · FRRouting 10.x (VyOS 1.5) · Linux kernel 6.6 LTS (VyOS 1.5 base) · strongSwan 5.9.x (IPsec) · WireGuard 1.0.x (kernel module + userspace tooling) · 2026-08-15

Not yet marked complete on this device.

A tunnel wraps an inner packet inside an outer packet with additional headers (and, for encrypted tunnels, additional trailers and authentication overhead). The wire-level MTU is unchanged (still 1500 bytes for Ethernet), but the available space for the inner packet is reduced by the tunnel overhead. For an application that sends 1500-byte packets, the tunnel must reduce the inner MTU to 1500 - tunnel overhead or the packet will be silently dropped at the tunnel interface.

This lesson is the production reference for tunnel overhead on VyOS 1.5 LTS: the overhead of WireGuard, IPsec, GRE, and VXLAN; the MTU calculations; the configuration; and the validation.

The four tunnel types

A VyOS operator encounters four tunnel types. Each has a different overhead:

flowchart LR
  subgraph WIREGUARD["WireGuard"]
    W1["Inner IP packet<br/>(MTU - 32 or -80)"]
    W2["UDP header<br/>8 bytes"]
    W3["WireGuard header<br/>32 bytes"]
    W4["Outer IP header<br/>20 (v4) or 40 (v6) bytes"]
  end
  subgraph IPSEC["IPsec ESP"]
    I1["Inner IP packet<br/>(MTU - 50 to -66)"]
    I2["ESP header<br/>8 bytes"]
    I3["ESP trailer<br/>2-16 bytes"]
    I4["ICV<br/>8-16 bytes"]
    I5["Outer IP header<br/>20 bytes"]
  end
  subgraph GRE["GRE"]
    G1["Inner IP packet<br/>(MTU - 24)"]
    G2["GRE header<br/>4 bytes"]
    G3["Outer IP header<br/>20 bytes"]
  end
  subgraph VXLAN["VXLAN"]
    V1["Inner Ethernet frame<br/>(MTU - 50)"]
    V2["VXLAN header<br/>8 bytes"]
    V3["UDP header<br/>8 bytes"]
    V4["Outer IP header<br/>20 bytes"]
    V5["Outer Ethernet header<br/>14 bytes"]
  end

The total overhead for each tunnel type:

Tunnel typeOverhead (IPv4 outer)Overhead (IPv6 outer)Notes
WireGuard32 bytes (UDP 8 + WG 32 - 8 = 32, includes outer IP)80 bytes (UDP 8 + WG 32 + IPv6 40 = 80)Fixed overhead; modern cryptography
IPsec ESP (AES-GCM-128)50-58 bytes70-78 bytesIncludes ESP header, trailer, ICV; outer IP adds 20-40
IPsec ESP (AES-CBC + SHA-256)58-66 bytes78-86 bytesPadding overhead for CBC
GRE24 bytes44 bytes4-byte GRE header + 20-byte outer IPv4 header
VXLAN50 bytes70 bytes8-byte VXLAN + 8-byte UDP + 20-byte outer IPv4 + 14-byte outer Ethernet

The discipline: each tunnel type has a known overhead. The operator calculates the effective inner MTU as wire MTU - overhead and configures the tunnel interface accordingly.

WireGuard overhead

WireGuard uses a fixed 32-byte header and an 8-byte UDP encapsulation. The outer IP header adds 20 bytes (IPv4) or 40 bytes (IPv6). The total overhead is 32 bytes (IPv4) or 80 bytes (IPv6).

flowchart LR
  subgraph INNER["Inner IPv4 packet (1500 bytes)"]
    I1["IP header 20"] --> I2["Payload 1480"]
  end
  INNER --> WG
  WG["WireGuard"]
  WG --> OUTER
  subgraph OUTER["Outer IPv4 packet (1532 bytes)"]
    O1["Outer IP 20"] --> O2["UDP 8"] --> O3["WireGuard 32"] --> O4["Inner IP 20"] --> O5["Payload 1452"]
  end

Wait — the inner packet is 1500 bytes (L3 MTU). The outer packet is 1500 + 32 = 1532 bytes. The wire MTU is 1500 bytes. So the outer packet (1532 bytes) does NOT fit on the wire.

The fix: configure the WireGuard interface’s inner MTU to 1500 - 32 = 1468 bytes. Then the inner packet is 1468 bytes; the outer packet is 1468 + 32 = 1500 bytes; the outer packet fits on the wire.

configure
set interfaces wireguard wg0 mtu 1468
commit

The operator validates:

show interfaces wireguard wg0
# MTU 1468

For IPv6 outer, the overhead is 80 bytes; the inner MTU is 1500 - 80 = 1420 bytes:

configure
set interfaces wireguard wg0 mtu 1420
commit

IPsec overhead

IPsec ESP (Encapsulating Security Payload) adds more overhead than WireGuard:

  • ESP header — 8 bytes (SPI 4, Sequence 4)
  • ESP trailer — 2 bytes minimum (padding + pad-length + next-header), more with CBC mode
  • ICV (Integrity Check Value) — 8 bytes (AES-GCM-128 truncated) or 12-16 bytes (SHA-256)
  • Outer IP header — 20 bytes (IPv4) or 40 bytes (IPv6)

For AES-GCM-128 (the most common modern cipher):

  • IPv4 outer: 20 + 8 + 2 + 8 = 38 bytes overhead
  • IPv6 outer: 40 + 8 + 2 + 8 = 58 bytes overhead

For AES-CBC + SHA-256 (legacy):

  • IPv4 outer: 20 + 8 + 16 (padding to 16) + 12 (SHA-256 ICV) = 56 bytes overhead
  • IPv6 outer: 40 + 8 + 16 + 12 = 76 bytes overhead

The discipline: IPsec overhead varies by cipher. The operator must know which cipher is configured (show ipsec status or swanctl --list-sas) and calculate the inner MTU accordingly.

A conservative default for IPsec with AES-GCM-128:

configure
set interfaces vti vti0 mtu 1438
# 1500 - (20 outer IP + 8 UDP-NAT-T or 0 + 8 ESP + 2 trailer + 8 ICV) ≈ 1454 for AES-GCM-128 without NAT-T
# 1438 leaves margin for NAT-T (UDP 8) and ICV variation
commit

The VTI (Virtual Tunnel Interface) is the IPsec tunnel interface. The operator sets the VTI MTU to the calculated inner MTU.

GRE overhead

GRE (Generic Routing Encapsulation) adds a 4-byte GRE header plus the outer IP header (20 bytes IPv4, 40 bytes IPv6). Total overhead: 24 bytes (IPv4) or 44 bytes (IPv6).

flowchart LR
  subgraph GRE_INNER["Inner IP packet"]
    GI["IP header + payload<br/>up to 1476 bytes (IPv4) or 1456 bytes (IPv6)"]
  end
  GRE_INNER --> GRE_TUNNEL["GRE"]
  GRE_TUNNEL --> GRE_OUTER
  subgraph GRE_OUTER["Outer IP packet"]
    GO["Outer IP header 20 (IPv4) or 40 (IPv6) + GRE header 4 + Inner IP packet"]
  end

The GRE interface is configured as a tunnel interface:

configure
set interfaces tunnel tun0 mtu 1476
# IPv4 outer: 1500 - 24 = 1476
# IPv6 outer: 1500 - 44 = 1456
commit

GRE does not encrypt; the inner packet is visible on the wire (inside the GRE wrapper). The discipline: GRE adds 24 bytes of overhead (IPv4); encrypt with IPsec for confidentiality.

VXLAN overhead

VXLAN (Virtual Extensible LAN) wraps an entire Ethernet frame inside a UDP packet. The overhead:

  • Outer Ethernet header — 14 bytes
  • Outer IP header — 20 bytes (IPv4) or 40 bytes (IPv6)
  • UDP header — 8 bytes
  • VXLAN header — 8 bytes

Total overhead: 50 bytes (IPv4) or 70 bytes (IPv6).

flowchart LR
  subgraph VX_INNER["Inner Ethernet frame (1522 bytes for 1500 MTU)"]
    VI["DA 6 + SA 6 + EtherType 2 + VLAN 4 + Payload 1500 + FCS 4"]
  end
  VX_INNER --> VX_WRAP["VXLAN"]
  VX_WRAP --> VX_OUTER
  subgraph VX_OUTER["Outer packet (1572 bytes)"]
    VO["Outer Ethernet 14 + Outer IP 20 + UDP 8 + VXLAN 8 + Inner Ethernet 1522 = 1572"]
  end

The inner MTU calculation:

  • Inner L3 MTU: 1500 bytes
  • Inner L2 frame: 1518 bytes (untagged) or 1522 bytes (tagged)
  • VXLAN overhead: 50 bytes
  • Outer L2 frame: 1568 bytes (untagged) or 1572 bytes (tagged)

For a standard 1500-byte inner MTU, the outer frame is 1572 bytes — does not fit in a 1518-byte wire MTU.

The fix: configure the VXLAN interface’s MTU to 1500 - 50 = 1450 bytes (for IPv4 outer):

configure
set interfaces vxlan vxlan0 mtu 1450
commit

For IPv6 outer, the MTU is 1500 - 70 = 1430 bytes.

Production failure modes

The tunnel overhead failure modes the operator encounters:

  • Oversized inner packet. Application sends 1500-byte packets; tunnel interface has default MTU; outer packet exceeds wire MTU; packet dropped. Fix: configure the inner MTU to wire MTU - overhead.
  • Asymmetric tunnel MTU. One end has MTU 1468 (WireGuard IPv4); the other end has MTU 1420 (default, IPv6 outer). Packets that fit on the small-MTU end are dropped on the large-MTU end. Fix: align the MTU on both ends.
  • MTU change breaks PMTUD. The tunnel MTU is set lower than the path MTU; PMTUD discovers the path MTU; the tunnel cannot fragment; packets are dropped. Fix: enable MSS clamping (Part LI-04) so TCP segments fit the tunnel MTU.
  • Tunnel MTU larger than the underlying interface. The operator configures tunnel MTU 1500 on a tunnel whose underlying interface is MTU 1400 (a low-MTU uplink). The tunnel packets exceed the underlying MTU; dropped at the underlying interface. Fix: tunnel MTU must be ≤ underlying MTU - overhead.
  • GRE + IPsec double overhead. The operator stacks GRE inside IPsec (common for route-based VPN with dynamic routing). The effective overhead is GRE (24) + IPsec (50-66) = 74-90 bytes. The inner MTU must account for both. Fix: tunnel MTU = wire MTU - 90 = 1410 bytes.

Rollback

Tunnel MTU changes are simple to roll back:

  • commit-confirm 5 — auto-rollback restores the previous MTU if the tunnel does not pass traffic.
  • Restore the default — for WireGuard, the kernel’s auto-calculation is the canonical default; for IPsec VTI, the default depends on the cipher; for GRE, the default is the underlying MTU - 24; for VXLAN, the default is the underlying MTU - 50.

Production discipline

Cross-course references

  • Part LI-01 (LI-VyOS-MTU / MTU basics) covers the canonical MTU values (1500, 9000, 1280, 802.1Q overhead).
  • Part LI-03 (LI-VyOS-MTU / PMTUD) covers Path MTU Discovery and ICMP Frag Needed.
  • Part LI-04 (LI-VyOS-MTU / MSS clamping) covers the TCP MSS adjustment that fits TCP segments into the tunnel MTU.
  • Part LI-05 (LI-VyOS-MTU / MTU troubleshoot) covers the operational diagnostic for MTU mismatches.
  • Part XLI (XLI-VyOS-WireGuard) covers WireGuard configuration.
  • Part XLII (XLII-VyOS-IPsec) covers IPsec configuration, including the cipher selection that affects overhead.

Quiz

Knowledge check · 4 questions

  1. Q1. An operator configures a WireGuard tunnel with IPv4 outer encapsulation. What is the inner MTU for a 1500-byte wire MTU?

  2. Q2. IPsec tunnel overhead is a fixed 50 bytes regardless of the cipher used.

  3. Q3. An operator stacks GRE inside IPsec (a common pattern for route-based VPN with dynamic routing). What is the inner MTU for a 1500-byte wire MTU?

    R1 has a GRE tunnel (`tun0`) inside an IPsec tunnel (`vti0`). The application sends a 1500-byte packet. The GRE interface adds 24 bytes (IPv4 outer); the IPsec interface adds 50 bytes (AES-GCM-128). The total overhead is 24 + 50 = 74 bytes. The inner MTU must be `1500 - 74 = 1426` bytes.

  4. Q4. An operator is configuring VXLAN on a data-centre backbone that supports jumbo frames (MTU 9000). What is the inner MTU for VXLAN with IPv4 outer on this backbone?

    R1 is a data-centre ToR (top-of-rack) router. The backbone supports MTU 9000. The operator is configuring VXLAN for tenant isolation. The VXLAN overhead is 50 bytes (IPv4 outer). The inner MTU must account for the overhead.

Passing score: 75%. Answers are checked in this browser.