Control-plane protection — the input chain, BGP TTL security (GTSM), and authentication on every peer

The control plane is the part of the router that runs the routing protocol daemons and the management services. Unlike the forwarding path, it runs on the CPU. A flood of packets addressed to the router — protocol packets, SSH attempts, SNMP polls, ICMP echoes — can exhaust the CPU and take the routing protocols down with it, without a single forwarded packet being involved.

Three mechanisms compose the production defence, and on VyOS 1.5 they live in three different places:

1. Rate limiting on the input path. Not a “CoPP” feature — see the next section — but the firewall ipv4 input filter chain plus firewall global-options.
2. GTSM (RFC 5082) rejects BGP packets from a peer that is not the configured number of hops away, which locks the session to the link you expect it on.
3. Authentication on every peer rejects protocol packets that do not carry the right key.

The control-plane flooding threat

A router’s CPU handles protocol packets (BGP, OSPF, IS-IS), management traffic (SSH, HTTPS, SNMP) and ICMP. That capacity is finite. Enough packets aimed at it and the routing protocols miss their hellos and the adjacencies drop — which is a far more expensive outcome than the flood itself.

The threat model:

• An attacker with reach to a transit link, flooding the router with protocol packets.
• A misconfigured or looping peer sending far more than its expected rate — statistically the commonest cause, and it is not an attack at all.
• A scanner enumerating TCP/22 and TCP/443 on every address it can find.

flowchart LR
  A[Flood source] -->|TCP/22 SYN| R[Router CPU]
  A -->|OSPF Hello| R
  A -->|TCP/179 SYN| R
  R -->|CPU saturated| FAIL[Hellos missed, adjacencies drop]

VyOS does not have a CoPP subsystem

Global options first

Several protections are one-liners and apply everywhere:

set firewall global-options syn-cookies 'enable'
set firewall global-options source-validation 'strict'
set firewall global-options broadcast-ping 'disable'
set firewall global-options ip-src-route 'disable'
set firewall global-options log-martians 'enable'
set firewall global-options state-policy established action 'accept'
set firewall global-options state-policy related action 'accept'
set firewall global-options state-policy invalid action 'drop'

syn-cookies is the specific answer to a SYN flood aimed at the router’s own listening sockets. source-validation strict is reverse-path filtering, which discards packets arriving on an interface the router would not use to reach their source — cheap, and it removes most spoofed traffic before any rule runs. The state policy means established sessions are accepted globally, so your input rules only have to think about NEW.

Rate limiting the input chain

set firewall ipv4 input filter default-action 'drop'
set firewall ipv4 input filter default-log

# BGP from the known peer, rate-limited on session setup only
set firewall ipv4 input filter rule 10 action 'accept'
set firewall ipv4 input filter rule 10 description 'BGP from 10.0.0.1'
set firewall ipv4 input filter rule 10 protocol 'tcp'
set firewall ipv4 input filter rule 10 destination port '179'
set firewall ipv4 input filter rule 10 source address '10.0.0.1'
set firewall ipv4 input filter rule 10 state 'new'
set firewall ipv4 input filter rule 10 limit rate '5/second'
set firewall ipv4 input filter rule 10 limit burst '10'

# OSPF from the internal interface only
set firewall ipv4 input filter rule 20 action 'accept'
set firewall ipv4 input filter rule 20 description 'OSPF on core links'
set firewall ipv4 input filter rule 20 protocol 'ospf'
set firewall ipv4 input filter rule 20 inbound-interface name 'eth1'

# SSH from management only, with brute-force damping
set firewall ipv4 input filter rule 30 action 'accept'
set firewall ipv4 input filter rule 30 protocol 'tcp'
set firewall ipv4 input filter rule 30 destination port '22'
set firewall ipv4 input filter rule 30 source address '10.99.0.0/24'
set firewall ipv4 input filter rule 30 state 'new'
set firewall ipv4 input filter rule 30 limit rate '10/minute'

# ICMP, capped
set firewall ipv4 input filter rule 40 action 'accept'
set firewall ipv4 input filter rule 40 protocol 'icmp'
set firewall ipv4 input filter rule 40 limit rate '20/second'
set firewall ipv4 input filter rule 40 limit burst '40'

GTSM — BGP TTL security

GTSM (RFC 5082) is a defence against an attacker who is not where your peer is. The mechanism is arithmetic on the TTL:

• The BGP speaker sends with TTL 255.
• The receiver accepts only packets whose received TTL is high enough to be consistent with the peer being at most the configured number of hops away.

TTL is decremented by every router that forwards the packet. A directly-connected peer’s packet is not forwarded by anything, so it arrives with TTL 255. A source three routers away arrives with 252. An attacker somewhere on the Internet cannot raise the TTL above 255 to compensate — the field is capped — so they cannot forge proximity.

set protocols bgp system-as '64512'
set protocols bgp neighbor 10.0.0.1 remote-as '64998'
set protocols bgp neighbor 10.0.0.1 ttl-security hops '1'

The hop count is the maximum distance you will accept, not a TTL value. hops 1 means “this peer is directly connected”; a loopback-to-loopback iBGP peer two routers away needs hops 3 or whatever the real path length is, with a little headroom if the path can change.

Enable it on both ends. GTSM configured on one side only protects that side, and does nothing for the peer.

Authentication on every peer

# BGP — TCP MD5 on the session
set protocols bgp neighbor 10.0.0.1 password 'SECRET-BGP-KEY'

# OSPF — MD5 HMAC on the interface, and on the area
set protocols ospf area 0 authentication md5
set protocols ospf interface eth1 authentication md5 key-id '1' md5-key 'SECRET-OSPF-KEY'

# IS-IS — interface password
set protocols isis interface eth1 password plaintext-password 'SECRET-ISIS-KEY'

The BGP password node is TCP-MD5 (RFC 2385). It is old, it is what almost every peer will agree to, and it is enormously better than nothing: it moves the attack from “inject a TCP segment into a known session” to “know the key”. OSPF’s authentication md5 is HMAC-MD5 over the protocol packet, configured per interface with a key ID so that keys can be rolled.

The composition

flowchart TB
  PKT[Packet addressed to the router] --> FW{input filter:<br/>source, port, rate}
  FW -->|no match / over rate| DROP1[Dropped by default-action]
  FW -->|accepted| K{kernel socket checks:<br/>min TTL, TCP-MD5}
  K -->|TTL too low| DROP2[Dropped: GTSM]
  K -->|digest mismatch| DROP3[Dropped: authentication]
  K -->|passes| DAEMON[bgpd / ospfd processes it]

Each layer catches a different class of unwanted packet, and none of them substitutes for another. The input chain bounds the volume; GTSM bounds where a peer may be; authentication bounds who a peer may be.

Failure modes

The input chain locked out the operator

set firewall ipv4 input filter default-action 'drop' committed before the SSH accept rule exists ends your session and leaves a console-only router. This is the single most common self-inflicted outage in this lesson.

Build the accept rules first, commit them, verify you can still reach the box on a second session, and only then set the default action — under commit-confirm.

The rate limit is throttling a legitimate protocol

A limit rate that is too tight does not slow the traffic; it makes the rule stop matching and the excess falls to default-action 'drop'. During a BGP full-table reconvergence or an OSPF flap storm, the legitimate rate is much higher than the steady-state rate you sized against.

Diagnostic:

show firewall ipv4 input filter rule 10
show firewall statistics
show log firewall ipv4 input filter

Compare the rule’s counter against the chain’s default-action counter. A default-action counter rising in step with a protocol event, while the specific accept rule plateaus, is the signature. The fix is to raise the rate and the burst — and to confirm the rule matches state 'new', so that steady-state protocol traffic was never subject to the limit in the first place.

GTSM configured with the wrong hop count

ttl-security hops 1 on a peer that is actually reached over two routers means every packet arrives with a TTL below the acceptable floor and the session never leaves Idle or Connect.

Diagnostic:

show bgp neighbor 10.0.0.1
show log | match bgp
sudo tcpdump -ni eth0 'tcp port 179' -v

The -v matters: it prints the TTL of each packet, which is the measurement that settles the argument. Count the real hops and set hops to that number. If the peer is reached by a path whose length can change, size for the longest path — GTSM’s protection degrades gracefully with a larger hop count, but a hop count that is too small is a hard outage.

GTSM configured on one side only

The session establishes and everything looks fine, because GTSM only filters what you receive. Nothing warns you that the peer has no protection. It is a per-side control and belongs in the peering template, not in an individual change.

Authentication key mismatch

The session does not establish, and with TCP-MD5 the failure is unusually opaque: the segments are discarded by the TCP stack, so bgpd sees a connection that never completes rather than an authentication error. sudo tcpdump -ni eth0 'tcp port 179' showing repeated SYNs with no SYN/ACK, on a peer that was working before someone touched the key, is the tell.

Rollback

compare
delete firewall ipv4 input filter rule 10 limit
commit

Deleting the limit node from one rule is the surgical fix for a too-tight rate and takes effect immediately.

delete protocols bgp neighbor 10.0.0.1 ttl-security
delete protocols bgp neighbor 10.0.0.1 password
commit

Removing GTSM or the password will bring a stuck session back, at the cost of the protection — acceptable as a diagnostic step, not as a resting state. Note that changing either one resets the session.

rollback N reverts the whole configuration but reboots the router. On a box whose control plane you are actively debugging, prefer the targeted delete commands above, and make every input-chain change under commit-confirm.

Nothing in the commit validator checks that your rates are above the legitimate protocol rate. That verification is yours, and a rate that is too tight is a silent failure that only shows up during the event you built it for.

Production discipline

Cross-course references

• LIII-VyOS-Security (vyos-liii-01-routing-protocol-authentication) covers the authentication mechanisms in detail.
• XXIV-VyOS-BGPSession (vyos-xxiv-05-ebgp-multihop) covers the multi-hop configuration that GTSM excludes.
• XXXVII-VyOS-Firewall covers the base chains, the state policy and the limit match used here.
• XLVII-VyOS-MgmtHardening covers the management-plane hardening that complements this.
• The Linux course’s XXV-Linux-Firewall covers the underlying netfilter hooks the input chain is built on.

Quiz