MTU and fragmentation troubleshoot — ping -M do -s, tracepath, ICMP filtering

“Large packets fail” is one of the most common operator reports and one of the hardest to diagnose without a structured method. The reports vary: HTTPS uploads of 100 KB stall; SSH works but SCP hangs on large files; a database backup fails; a video call drops after a few minutes. The common thread is that the application sends TCP segments larger than the path can forward.

This lesson is the operational diagnostic for MTU and fragmentation on VyOS 1.5 LTS: the ping -M do -s and tracepath tools, the diagnostic ladder, and the canonical fix for each outcome.

The diagnostic ladder

flowchart TD
  S["Symptom<br/>(large packets fail)"]
  S --> Q1{Does ping work at default size?}
  Q1 -->|no| N1["Not an MTU issue<br/>investigate connectivity"]
  Q1 -->|yes| Q2{Does ping -M do -s 1472 work?}
  Q2 -->|yes| N2["Path MTU >= 1500<br/>no MTU issue"]
  Q2 -->|hangs| Q3{Does ping -M do -s 1300 work?}
  Q3 -->|yes| Q4["Path MTU 1300-1500<br/>find bottleneck"]
  Q3 -->|hangs| Q5{Does ping -M dont -s 1472 work?}
  Q5 -->|yes| Q6["PMTUD black hole<br/>ICMP filtered"]
  Q5 -->|hangs| Q7["Path broken<br/>investigate routing"]
  Q4 --> T1["tracepath to find bottleneck"]
  Q6 --> T2["tcpdump for ICMP"]

The ladder works from large to small, with progressively more diagnostic information.

Tool 1: ping -M do -s

The canonical MTU diagnostic. The -M do flag sets DF (Don’t Fragment); the -s flag sets the payload size (the IP packet size is the payload plus 8 for the ICMP echo header plus 20 for the IPv4 header).

$ ping -M do -s 1472 198.51.100.1
PING 198.51.100.1 (198.51.100.1) 1472(1500) bytes of data.
From 10.0.0.1 icmp_seq=1 Frag needed and DF set (mtu = 1450)

The Frag needed and DF set line is the ICMP Fragment Needed message, and it is good news: something on the path is telling you the truth. The mtu = 1450 names the bottleneck, and the source now knows the path MTU is 1450 rather than the 1500 it assumed.

Read the value carefully. A router that reports a bottleneck equal to the size you sent is not reporting a bottleneck at all — that combination cannot happen, and if you think you are seeing it you are reading a stale line from an earlier probe.

A different outcome:

$ ping -M do -s 1472 198.51.100.1
PING 198.51.100.1 (198.51.100.1) 1472(1500) bytes of data.
^C
--- 198.51.100.1 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2000ms

The ping hangs; no ICMP reply. This is the PMTUD black hole signature.

The operator’s first step: try a smaller size to confirm the path works at smaller MTU:

$ ping -M do -s 1300 198.51.100.1
PING 198.51.100.1 (198.51.100.1) 1300(1328) bytes of data.
1308 bytes from 198.51.100.1: icmp_seq=1 ttl=64 time=0.5 ms
1308 bytes from 198.51.100.1: icmp_seq=2 ttl=64 time=0.4 ms

Path MTU is at least 1328 bytes (1300 + 8 ICMP echo + 20 IP header). The path works at 1300 but not at 1472.

The operator bisects to find the exact bottleneck:

$ for size in 1300 1400 1450 1472; do
    echo "=== Testing size $size ==="
    ping -M do -s $size -c 3 198.51.100.1
done

The output shows the boundary between success and failure:

=== Testing size 1300 ===
3 packets transmitted, 3 received, 0% packet loss

=== Testing size 1400 ===
3 packets transmitted, 3 received, 0% packet loss

=== Testing size 1450 ===
From 10.0.0.1: Frag needed and DF set (mtu = 1450)

=== Testing size 1472 ===
From 10.0.0.1: Frag needed and DF set (mtu = 1450)

The path MTU is 1450 bytes. The bottleneck is at MTU 1450 (not 1500). The operator now knows the exact value to configure.

Tool 2: tracepath

tracepath walks the path hop-by-hop and reports the MTU at each hop:

$ tracepath 198.51.100.1
 1?: [LOCALHOST]                                         pmtu 1500
 1:  10.0.0.1                                             0.5ms
 1:  10.0.0.1                                             0.4ms
 2?: 192.0.2.1                                            0.6ms
 2:  192.0.2.1                                            0.5ms pmtu 1450
 3:  198.51.100.1                                         1.2ms
 3:  198.51.100.1                                        12.4ms reached
     Resume: pmtu 1450

The output shows:

• Hop 1 (10.0.0.1): MTU 1500
• Hop 2 (192.0.2.1): MTU 1450 (this is the bottleneck)
• Hop 3 (198.51.100.1): destination

The path MTU is 1450 bytes (the minimum MTU along the path).

A different outcome (PMTUD black hole):

$ tracepath 198.51.100.1
 1?: [LOCALHOST]                                         pmtu 1500
 1:  10.0.0.1                                             0.5ms
 1:  10.0.0.1                                             0.4ms
 2?: 192.0.2.1                                            0.6ms
 2:  192.0.2.1                                            0.5ms pmtu 1450
 3:  no reply
 3:  no reply
 3:  no reply
     Resume: pmtu 1450

The tracepath stops at hop 3 because hop 3 does not respond. The pmtu 1450 is still shown (because the earlier hop’s MTU was 1450). The destination is unreachable at the configured MTU; the cause is unknown (could be PMTUD black hole, could be actual unreachability).

Tool 3: tcpdump

When the diagnostic is ambiguous, the operator captures the ICMP on the wire. From VyOS operational mode that is monitor traffic, which is a wrapper around the same tcpdump:

monitor traffic interface eth0 filter 'icmp'

$ tcpdump -i eth0 -n icmp
12:34:56.789 IP 198.51.100.1 > 10.0.0.1: ICMP 198.51.100.1 unreachable - need to frag (mtu 1450), length 36

The unreachable - need to frag message is the ICMP Fragment Needed. The mtu 1450 is the bottleneck.

If tcpdump shows no ICMP packets at all when the source sends a large ping:

$ tcpdump -i eth0 -n icmp
# No output

The path is silently dropping the large packet without sending ICMP. This is the PMTUD black hole signature.

The operator then checks the local firewall. On VyOS 1.5 that is two questions, not one, and skipping the second is the usual reason this step produces a false clean bill of health:

show firewall ipv4 name WAN-IN
show firewall ipv4 forward filter

The first shows the named ruleset and its per-rule counters — look for a rule permitting ICMP type 3 code 4, and read its counter rather than its existence. The second shows the base hook, and answers the question the first cannot: is anything jumping to WAN-IN at all? A named ruleset with a perfect ICMP rule and no action jump pointing at it filters nothing and permits nothing, because no packet reaches it.

The rule you are looking for, if it is missing, is this — and the hook matters, because a fragmentation-needed message travelling back to a host behind the router is transit traffic:

set firewall ipv4 forward filter rule 6 action 'accept'
set firewall ipv4 forward filter rule 6 description 'ICMP frag-needed — required for PMTUD'
set firewall ipv4 forward filter rule 6 protocol 'icmp'
set firewall ipv4 forward filter rule 6 icmp type '3'
set firewall ipv4 forward filter rule 6 icmp code '4'

If the local firewall already allows ICMP Fragment Needed and the black hole persists, the drop is upstream — an ISP, a partner network, a transit provider — and you have run out of things you can configure. Ask them whether they filter ICMP type 3 code 4, and expect the answer to take longer than the incident. MSS clamping is the fix you can apply unilaterally in the meantime, which is why it exists.

Tool 4: ping -M dont -s (fragmentation test)

The -M dont flag disables DF; the source fragments the packet and forwards it. The ping should succeed at any size (subject to the underlying MTU and the path’s ability to deliver fragments):

$ ping -M dont -s 8000 198.51.100.1
PING 198.51.100.1 (198.51.100.1) 8000(8028) bytes of data.
8008 bytes from 198.51.100.1: icmp_seq=1 ttl=64 time=1.0 ms

The ping succeeds because the source fragments before transmitting. On a 1500-byte link each fragment carries at most 1480 bytes of the original payload (1500 minus the 20-byte IP header it has to repeat), so the 8008 bytes of ICMP-plus-data become five full fragments of 1480 and a sixth carrying the remaining 608 — six IP packets, five of them 1500 bytes on the wire and one of 628. The destination reassembles them and replies.

Note what this proves and what it does not. It proves the path forwards fragments and that the far end reassembles them. It says nothing about whether the path forwards a non-fragmented packet of the same total size, which is the case every real application actually exercises.

If -M dont succeeds but -M do hangs at the same size, the issue is PMTUD, not routing:

$ ping -M dont -s 8000 198.51.100.1  # Succeeds
$ ping -M do -s 8000 198.51.100.1    # Hangs

The path can deliver fragments but cannot deliver a non-fragmented large packet. The cause is a router that drops large packets and does not send ICMP Fragment Needed.

The four diagnostic outcomes

The operator’s ladder produces one of four outcomes:

flowchart TD
  S["Diagnostic"]
  S --> O1["Outcome 1: ping -M do -s <large> succeeds<br/>(path MTU >= wire MTU)"]
  S --> O2["Outcome 2: ping -M do -s <large> gets ICMP Frag Needed<br/>(PMTUD works, path MTU known)"]
  S --> O3["Outcome 3: ping -M do -s <large> hangs<br/>ping -M dont -s <large> succeeds<br/>(PMTUD black hole, path works)"]
  S --> O4["Outcome 4: ping -M dont -s <large> hangs<br/>(path is actually broken)"]

For each outcome, the canonical fix:

1. Outcome 1: large succeeds. No MTU issue. Look elsewhere (DNS, routing, application).
2. Outcome 2: large gets ICMP. PMTUD works; the source learns the path MTU. If the application still fails, the application does not use PMTUD (e.g., NFS over UDP, custom application). Fix: enable PMTUD in the application or use MSS clamping.
3. Outcome 3: PMTUD black hole. The path can deliver fragments but cannot deliver non-fragmented large packets because ICMP is filtered. Fix: enable MSS clamping at the boundary.
4. Outcome 4: path broken. The large ping hangs even with -M dont. The path is genuinely unreachable at that size; routing or ACL is the issue, not MTU. Fix: investigate routing and ACL.

Production failure modes

The MTU troubleshooting failure modes the operator encounters:

• MSS clamping on the wrong interface. The clamp sits on the LAN interface while the constrained hop is the tunnel, so segments are sized against a limit that is not the binding one. Fix: put ip adjust-mss on the interface whose MTU is the actual constraint, and then prove it with a capture of the SYN rather than by re-reading the config.
• Only half the address families clamped. ip adjust-mss and ipv6 adjust-mss are separate nodes with separate values. Setting one leaves a dual-stack router where v4 works and v6 stalls on large transfers, which reads to users as “the site is broken for some people”. Fix: set both.
• MSS clamping too aggressive. MSS is set below the actual path MTU. Throughput suffers, because a smaller segment carries the same 40 bytes of header. Fix: raise the value to the path MTU minus 40 (20 IP + 20 TCP), or use clamp-mss-to-pmtu and let the kernel derive it per route.
• ICMP filtering on upstream. The operator permits ICMP Fragment Needed in the local firewall and PMTUD still fails, because the drop was never local. Fix: MSS clamping, which does not depend on ICMP surviving anything.
• Tunnel MTU left at the default. The tunnel comes up and forwards traffic at an MTU that does not account for the encapsulation overhead of this path — the default is a guess about a typical underlay, not a measurement of yours. Fix: measure the underlay with a DF probe and set mtu on the tunnel to what you measured.
• Asymmetric tunnel MTU. One end has MTU 1412 and the other still has the default. Traffic works in one direction and stalls in the other, which produces the “it’s only slow when uploading” report. Fix: set both ends to the same value from the same measurement.
• Tracepath hangs at hop N. The destination is not reachable, or that hop simply does not generate the ICMP tracepath needs. Either way it is not evidence of an MTU problem. Fix: confirm reachability first, then re-run the MTU ladder.

Rollback

MTU diagnostic changes are about analysis, not rollback. The discipline:

• Save diagnostic outputs to a file or ticket: ping -M do -s <size> results, tracepath output, tcpdump captures.
• Document the path MTU in the operator’s MTU inventory.
• Apply the fix (MSS clamping, firewall rule, MTU change) and validate with the same diagnostic.

Production discipline

Cross-course references

• Part LI-01 (LI-VyOS-MTU / MTU basics) covers the canonical MTU values.
• Part LI-02 (LI-VyOS-MTU / tunnel overhead) covers the overhead calculations for tunnels.
• Part LI-03 (LI-VyOS-MTU / PMTUD) covers Path MTU Discovery.
• Part LI-04 (LI-VyOS-MTU / MSS clamping) covers the canonical fix when PMTUD fails.
• Part LI-06 (LI-VyOS-MTU / MTU validation) covers end-to-end MTU validation for production routers.
• Part XXXVII (XXXVII-VyOS-Firewall) covers firewall configuration, including ICMP rules.

Quiz