Tunnel Up, No Routes Learned

Reported symptoms

Branch 7 reaches HQ over a WireGuard tunnel built in March. The tunnel subnet is 10.10.10.0/30, HQ holds .1 and the branch holds .2, and the design on file says the branch prefixes are carried by OSPF across the tunnel in the same area as the rest of the estate.

On Thursday the branch commissioned a second VLAN, 192.168.17.0/24, for a new production line. It has never been reachable from HQ. The branch’s original LAN, 192.168.7.0/24, works and has worked since day one.

The evidence that arrived on the ticket over the first hour all pointed away from the tunnel:

• The tunnel is healthy on every measure anybody has. show interfaces wireguard wg0 summary reports a handshake seconds old at both ends with bytes moving both ways, and ping 10.10.10.2 from HQ is clean.
• The OSPF configuration on wg0 is the same stanza that works on six other branches. Those six reach HQ over ethernet handoffs and their adjacencies have never given trouble.
• A traceroute from HQ into 192.168.7.0/24 completes, which reads as proof that routing over the tunnel works.

Two facts arrived later and are worth more than the rest combined. First, show ip ospf neighbor is empty on both routers — and so is the capture that was archived when the link was commissioned in March, which nobody read at the time. Second, on Friday an engineer applied the change that every search result recommends for OSPF over a tunnel, setting the interface network type to point-to-point, and it changed nothing at all. It is still in the configuration.

Evidence provided

vyos@edge1:~$ show interfaces wireguard wg0 summary

interface: wg0
public key: 6PN0kRVL2Yy1kW6h3hAcQOx2m6QqQZ5Y7uZ0mM8wXjk=
listening port: 51820

peer: rSuuvkAuSoyqvogoA5OJjYBbJKKaYS13pWs4K69PTOE=
endpoint: 203.0.113.47:51820
allowed ips: 10.10.10.0/30, 192.168.7.0/24, 192.168.17.0/24
latest handshake: 34 seconds ago
transfer: 41.28 MiB received, 63.91 MiB sent

vyos@edge1:~$ ping -c 3 10.10.10.2

PING 10.10.10.2 (10.10.10.2) 56(84) bytes of data.
64 bytes from 10.10.10.2: icmp_seq=1 ttl=64 time=11.4 ms
64 bytes from 10.10.10.2: icmp_seq=2 ttl=64 time=11.1 ms
64 bytes from 10.10.10.2: icmp_seq=3 ttl=64 time=11.6 ms

--- 10.10.10.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss

vyos@edge1:~$ show ip ospf interface wg0

wg0 is up
Internet Address 10.10.10.1/30, Area 0.0.0.0
MTU mismatch detection: enabled
Router ID 10.255.0.1, Network Type POINTOPOINT, Cost: 10
Transmit Delay is 1 sec, State Point-To-Point, Priority 1
Timer intervals configured, Hello 10s, Dead 40s, Retransmit 5s
  Hello due in 3.418s
Neighbor Count is 0, Adjacent neighbor count is 0

vyos@edge1:~$ sudo tcpdump -nei wg0 -c 3 proto ospf

10:12:04.881207 ip: 10.10.10.1 > 224.0.0.5: OSPFv2, Hello, length 44
10:12:14.882031 ip: 10.10.10.1 > 224.0.0.5: OSPFv2, Hello, length 44
10:12:24.882844 ip: 10.10.10.1 > 224.0.0.5: OSPFv2, Hello, length 44

vyos@br7:~$ sudo tcpdump -nei wg0 -c 3 proto ospf

tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on wg0, link-type RAW (Raw IP), snapshot length 262144 bytes
^C
0 packets captured

vyos@edge1:~$ ip link show wg0

7: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN
  link/none

vyos@edge1:~$ ip link show eth0

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
  link/ether 52:54:00:0e:33:a1 brd ff:ff:ff:ff:ff:ff

vyos@edge1:~$ show ip route 192.168.7.0/24

Routing entry for 192.168.7.0/24
Known via "static", distance 1, metric 0, best
Last update 22w4d ago
* 10.10.10.2, via wg0

Work the evidence before reading on

Two captures taken at the same moment disagree about whether a packet exists, and both of them are correct.

1. HQ sends a hello every ten seconds and the branch receives nothing. The tunnel demonstrably carries packets in both directions at the same moment — the ping proves it. What is different about the hello?
2. Read the destination address on the hello, then read the flag list on ip link show wg0, then read the same flag list on eth0, where this exact OSPF configuration works. Which word appears on one and not the other, and what does it govern?
3. The interface’s allowed-ips set is printed in the tunnel summary. Is the hello’s destination address inside it? What does WireGuard do with a packet that has been routed out of wg0 and is not covered by any peer’s set?
4. Friday’s change set the network type to point-to-point and produced no change in behaviour. Assume the engineer applied it correctly. What would that setting have to alter for it to have fixed this, and does it alter that?
5. show ip ospf neighbor was empty in March too. If the adjacency has never existed, what has been carrying 192.168.7.0/24 for five months, and why did the new VLAN behave differently from the old one on the very first day?

Then the question that resolves it: the tunnel is up, the peers are handshaking, and the /30 pings clean. Which of those three facts says anything at all about whether a multicast packet can cross this link?

Root cause

1. The interface is not a multicast-capable device

A Linux network device advertises what it can do in its flags, and the flag that matters here is the one that says the device can carry multicast. The WireGuard driver registers its interface as point-to-point and no-ARP, and never sets it. That is why ip link show wg0 prints four flags and ip link show eth0 prints a list containing MULTICAST, and it is the whole difference between the six branches whose adjacencies work and the one whose adjacency does not.

This is a property of the Linux driver rather than a VyOS setting or an FRR setting, so there is no configuration node anywhere that will change it. It also applies to the xfrm interface that VyOS presents as a VTI, which matters because “use IPsec instead” is the standard suggestion at this point in the investigation and it would have moved the problem nowhere.

2. On WireGuard, allowed-ips discards it before that even matters

WireGuard applies a second check underneath the routing table. Once the kernel has decided a packet leaves via wg0, cryptokey routing asks which peer’s allowed-ips set covers the destination; if none does, the packet is dropped.

The set on this tunnel is the /30 plus the two branch prefixes, which is a correct and ordinary set. It does not contain 224.0.0.5, so a hello addressed there is discarded on the way out, before the netdev flag question is reached.

Two independent obstacles, then, and it is worth keeping them apart, because removing one of them changes nothing observable. The captures show exactly that: the hello exists on the sending side, where tcpdump taps the interface before the driver disposes of it, and does not exist anywhere else.

3. point-to-point changes the adjacency model, not the destination

Friday’s change was applied correctly and could not have worked. The network type governs how the link is represented and whether a designated router is elected; it does not change where OSPF addresses its hellos. On VyOS 1.5 the only network type that stops OSPF using the multicast group and instead unicasts a hello to each statically configured neighbour is non-broadcast.

The reason this belief is so durable is that it is usually followed by success. On a genuine point-to-point link — a serial circuit, a GRE tunnel, which really is multicast-capable — setting point-to-point is the right thing to do, and it resolves several unrelated failure modes that also stop an adjacency forming. The change gets credit for those, and the one case it cannot fix produces no error message to contradict it.

4. A static route hid a design that has never worked

The adjacency did not break on Thursday. It has never existed.

In March the link was built, the OSPF stanza was written, and the branch LAN was made to work with a static route via the tunnel address. Traffic flowed, the link was signed off, and the archived show ip ospf neighbor capture — empty — was filed rather than read. For five months every packet to branch 7 followed a static route while the documentation described a dynamic one.

The new VLAN was the first prefix that depended on the mechanism instead of on the workaround, which is why a five-month-old defect presented as a Thursday outage.

Resolution

1. Restore service first and keep it separate from the design fix. A static route for 192.168.17.0/24 via 10.10.10.2 gets the production line working; it needs the prefix present in the peer allowed-ips on both ends or it forwards into a drop, and it needs an expiry on the ticket, because an unexpiring temporary static is exactly what hid this since March.
2. Record the negative result from Friday before touching the configuration again, then either revert the network type or set it deliberately as part of the fix. Do not leave a disproved change in place unexplained.
3. Decide the durable fix on design grounds, not incident grounds. BGP over the tunnel is unicast TCP to a configured address and never has to reason about multicast; it is the better standard for tunnelled sites and it is what the other tunnels in the estate should converge on.
4. Treat the BGP option as a change, not a repair. It brings an AS-numbering decision, a redistribution boundary, and a policy requirement — an eBGP session with no inbound and outbound policy attached can establish and exchange nothing, which would produce this exact ticket a second time.
5. If the estate is genuinely one OSPF domain and this tunnel is one more link in it, configure the network type as non-broadcast at both ends and add a static protocols ospf neighbor entry naming the far end tunnel address. The tunnel /30 must be inside allowed-ips, and the hello and dead intervals must match on both routers.
6. Refuse the 224.0.0.5 shortcut. Putting the multicast group into allowed-ips is a necessary condition for multicast OSPF over WireGuard and is not a verified sufficient one, because the interface is still not registered as multicast-capable. If you want to know, test it in a lab and read the destination address in a capture.
7. Refuse the migration to IPsec that will be proposed as the way around all of this. The interface VyOS presents as a VTI has the same limitation for the same driver-level reason.
8. Re-read the March sign-off and fix the record. The design document describes a mechanism that has never run; leaving that in place guarantees somebody rebuilds the same link the same way.

Verification

1. show ip ospf neighbor lists the far end in Full state at both routers, with the neighbour router-id you expect rather than one you have to look up afterwards.
2. A capture on the tunnel shows hellos addressed to the neighbour tunnel address rather than to 224.0.0.5. The destination address is what was wrong, so it is what you read; an adjacency that came up is not proof that you fixed the mechanism you think you fixed.
3. Remove the temporary static before verifying the prefix, or you will be verifying the static. A prefix that survives the deletion of the route covering it is the only evidence the protocol is carrying it.
4. show ip route 192.168.17.0/24 shows an OSPF route via the tunnel address out of wg0, and ip route get for a host inside that VLAN returns the same egress, because the kernel is what decides.
5. The check can fail. Shut the VLAN interface at the branch, watch the prefix leave the HQ table within the dead interval, and bring it back. A dynamic route nobody has watched appear and disappear is an assumption, and that assumption is what the March sign-off recorded as a fact.
6. Traffic is tested from hosts at both ends, not from the routers. A router pinging across a /30 was one of the three checks that made this look healthy for five months.
7. MTU and MSS handling on the tunnel is re-checked once real traffic crosses it. This path has been carrying almost nothing, and a working route is the first time the encapsulation overhead is exercised at production packet sizes.

Prevention

• Alert on OSPF interfaces with zero neighbours. An interface configured into an area and adjacent to nobody for longer than its dead interval is a defect whatever caused it, and that one check would have raised this in March instead of in August.
• Stop treating “up” as a health signal for WireGuard. The interface is up because it is configured. The signals that mean something are the age of the last handshake and an end-to-end probe over the link, and both belong in monitoring.
• Check the netdev flags before choosing a routing protocol for a link. ip link show is free, it answers whether multicast can work here at all, and no amount of protocol configuration overrides its answer.
• Verify each mechanism a design names, not just that traffic flows. This link passed its build review on the strength of a static route while one of its two documented mechanisms had never worked once.
• Standardise on BGP for tunnelled sites, and make OSPF over a tunnel an exception that carries a written reason, its static neighbour list, and a note explaining the network type — because the next engineer to tidy non-broadcast into point-to-point will break it silently and get no error to tell them so.
• Remove changes that did not work, and write down that they did not. A disproved fix left in a configuration is indistinguishable from a deliberate setting six months later.