BGP session troubleshooting

A BGP outage is not a request to change a random timer. It is a request to identify which contract failed. The session may be absent from the configuration, unable to open TCP, rejected during OPEN, held at the final KEEPALIVE, or repeatedly reset after it is Established. Each boundary produces a different state and a different evidence set.

The fastest safe workflow starts with the state, then checks the configuration, transport, protocol, policy, and forwarding result. The operator should preserve evidence before a reset because a hard reset can remove the exact Adj-RIBs-Out or route history needed to explain the incident.

The five-layer diagnostic model

Use five layers and move from the first failed boundary:

1. Configuration — Does the local and remote neighbor exist with the intended ASN, source, address family, and policy?
2. Transport — Can TCP 179 complete from the configured source to the destination and back?
3. Protocol — Does OPEN exchange, capability negotiation, hold time, and KEEPALIVE complete?
4. Policy and tables — Is the prefix accepted, selected, eligible for the peer, and installed in the FIB?
5. Forwarding and return — Can traffic and its reply traverse the selected path, including firewall and source policy?

flowchart TB
  S["BGP state"] --> C["Configuration check"]
  C --> T["TCP and route check"]
  T --> O["OPEN and capability check"]
  O --> P["BGP RIB and policy"]
  P --> F["FIB and traffic check"]
  F --> D["Diagnosis and rollback"]
  D --> S

A session that never reaches Established is likely in the first three layers. A session that reaches Established and has no prefixes may be healthy at transport but failed at policy or address-family activation. A session that has prefixes but no traffic may be healthy at BGP but failed at the next hop, return route, firewall, or data plane.

The first three commands

Run these in order and write the output to the incident record:

show bgp summary
show bgp neighbor 192.0.2.2
show configuration commands | match 'bgp|neighbor|update-source|password'

The summary answers whether the peer is known and what state it is in. The detailed neighbor output adds the remote AS, source interface, capabilities, timers, message counts, reset reason, and prefix counts. The active configuration proves what was committed, not what the operator remembers typing.

If the neighbor is absent from the summary, do not debug timers. The local peer is not committed, is in another VRF, is administratively shutdown, or the process has not loaded the new configuration.

Active: diagnose the TCP boundary

Active means the TCP connection is not completing reliably or the router is retrying. Start with the source and destination:

show route 192.0.2.1
show route 192.0.2.2
ping -c 3 192.0.2.2 source 192.0.2.1
ss -tnp | grep ':179'
tcpdump -ni any host 192.0.2.2 and tcp port 179

Read the capture by direction:

• no outbound SYN: local configuration, route, or socket source issue;
• outbound SYN and return SYN-ACK, then no final ACK: local or remote firewall, asymmetric state, or source policy;
• repeated SYN and RST: possible TCP-MD5 mismatch or an ACL rejecting the reset;
• SYN, SYN-ACK, ACK, OPEN, then a reset: transport is open and the BGP contract is failing;
• direct peer remains in Active: the source may be a loopback without the intended ebgp-multihop or disable-connected-check design.

Peer not configured and AS mismatch

A peer not configured on the remote side can produce the same local symptom as a local configuration omission. Compare both sides:

show configuration commands | match 'bgp|neighbor'
show bgp neighbor 192.0.2.2

Record the local remote-AS, the remote local-AS, the peer address, and the address family. The peer may be configured for a loopback while the local router points at its transit address. The two ends can both be reachable while they are not speaking to the expected BGP identity.

An AS mismatch is usually visible in OpenSent: TCP completed, one or both OPEN messages were exchanged, and the session then resets or remains out of Established. Correct one side at a time and verify the remote state. A same-AS configuration is not automatically an AS mismatch; it is a valid iBGP relationship with a different split- horizon behavior.

MD5 mismatch

TCP-MD5 is a transport contract. A mismatch can be caused by a typo, a secret changed on one side, a source address that is not part of the expected peer socket, or a device that applies the key to a different session. Do not paste the secret into the incident record. Compare a key identifier, rotation time, and the active peer configuration.

Useful evidence:

show bgp neighbor 192.0.2.2
show log protocol bgp
tcpdump -ni any host 192.0.2.2 and tcp port 179

A capture with SYN, SYN-ACK, ACK, and RST before a stable OPEN supports a transport authentication investigation. Restore the last known-good secret through the approved secret process, then verify the session and prefix counts.

TTL, multihop, and GTSM

The expected eBGP TTL is 1 for a directly connected peer. A routed loopback peer needs an explicit design:

set protocols bgp neighbor 192.0.2.2 update-source 192.0.2.1
set protocols bgp neighbor 192.0.2.2 ebgp-multihop 5

Check the source and return path with show route, ping source, and the packet capture. If the peer should be directly connected, remove the multihop setting and use its link address. If the peer is routed, document the maximum path and align the hop count with the actual routing design.

GTSM is a different contract:

set protocols bgp neighbor 192.0.2.2 ttl-security hops 1

The documented tree treats GTSM and eBGP multihop as mutually exclusive. A commit failure or repeated reset is evidence that the operator selected the wrong model, not a reason to set both.

OpenSent, OpenConfirm, and Established flaps

OpenSent points to OPEN parameters: AS, router ID, capabilities, or the address family. OpenConfirm points to the final KEEPALIVE, hold-time contract, or a reset after OPEN. Established flaps point to transport, policy, route, or timer problems that occur after the session is up.

A repeated Established-to-Idle cycle with a 90-second interval and a 180-second hold time is not evidence that 90 is the right timer. It means the peer stops receiving valid KEEPALIVE or UPDATE traffic after roughly half the hold interval. Inspect the return path, interface errors, congestion, and packet loss before changing the timer.

A reproducible incident worksheet

Before changing anything, capture:

Timestamp and change ticket:
Router and neighbor address:
Local AS and remote AS:
Current BGP state:
Negotiated hold time:
Source and update-source:
TTL or GTSM setting:
Authentication mechanism and key identifier:
Inbound and outbound policy:
Adj-RIBs-In and Loc-RIB evidence:
Adj-RIBs-Out and FIB evidence:
Packet-capture direction:
Rollback command and revision:

If the session is a remote access path, preserve an out-of-band path before changing the peer. Use commit-confirm when the platform and change window allow it. Do not let a troubleshooting command remove the only route to the operator.

Rollback discipline

A narrow rollback is safer than a full router reset:

• wrong neighbor address: delete and recreate the neighbor;
• wrong source: restore the prior source and verify its route;
• wrong TTL: remove the multihop or GTSM setting according to the selected design;
• wrong AS: restore the negotiated value on one side, then recheck the peer;
• wrong secret: restore the previous secret through the secret store;
• wrong timer: restore the prior value and observe the negotiated hold;
• wrong policy: remove the specific neighbor or address-family override, not the global baseline.

After the rollback, run the full evidence ladder again. A session that returns to Established but loses the expected prefix set is not a successful rollback.

Cross-course references

XXIII-VyOS-BGPFund covers the FSM, best path, and RIB layers. XXIV-VyOS-BGPSession provides the configuration, neighbor, timer, authentication, and multihop details. XXXI-VyOS-BGPTroubleshoot extends this into a full prefix and policy workflow, while XXXII-VyOS-BFD covers faster failure detection. The Linux course covers routes, TCP sockets, and packet capture, and the OPNsense course shows how to apply the same evidence method to FRR on another platform.

Quiz