BGP telemetry — BMP, RIB monitoring, and FRR streaming state

A “BGP is broken” report is not BGP until the operator has session state, RIB count, and update-burst evidence. The BGP RIB (Routing Information Base) is the canonical source of truth: every prefix the router has learned, every attribute, every path. The Adj-RIBs-In (per-peer received routes) and the Loc-RIB (the router’s best-path selection) are the two halves of the BGP state machine. The operator who cannot read those state structures in five minutes will spend an hour guessing why a peer is flapping, why a route is missing, or why a prefix is being preferred over another.

This lesson is the production reference for BGP telemetry on VyOS 1.5 LTS: the four transport options (BMP, FRR’s northbound JSON, gNMI, SNMP), what each one gives the operator, what each one costs, and the validation discipline that prevents the BGP telemetry layer from becoming the next source of incidents.

The four transports

flowchart LR
  subgraph ROUTER["VyOS 1.5 router (control plane)"]
    Z["FRR zebra<br/>Loc-RIB"]
    B["FRR bgpd<br/>Adj-RIBs-In"]
  end
  subgraph COLLECT["Collection plane"]
    BMP["BMP collector<br/>(OpenBMP, bmp-prometheus)"]
    FRR["FRR northbound<br/>JSON over HTTP"]
    GNMI["gNMI telemetry<br/>subscription"]
    SNMP["SNMP BGP4-MIB"]
  end
  subgraph CONSUME["Consumption plane"]
    TS["Time-series DB<br/>(Prometheus, InfluxDB)"]
    NMS["NMS / Grafana"]
  end
  B --> BMP
  Z --> FRR
  B --> GNMI
  Z --> SNMP
  BMP --> TS
  FRR --> TS
  GNMI --> TS
  SNMP --> NMS

BMP (BGP Monitoring Protocol, RFC 7854) is the canonical streaming protocol for BGP telemetry. BMP establishes a TCP session from the router to a collector, and the router streams every route advertisement, every withdrawal, every state change. The collector sees the full RIB at session establishment (the “Init” message) and every subsequent update as a deltas. The cost: the operator must run a BMP collector, and the collector must be sized to handle the router’s full RIB plus the update rate.

FRR’s northbound JSON is the ad-hoc inspection alternative. The show ip bgp summary json and show bgp ipv4 unicast json commands emit JSON output that the operator can parse with jq. The cost: the operator must poll (FRR does not push), and the polling rate is limited by the user’s patience (a 1-second poll of a 1,000,000-route RIB is slow).

gNMI telemetry is the streaming-evidence alternative for BGP. The operator subscribes to the BGP YANG model and receives state updates on a configurable cadence. The cost: gNMI requires the vyos-gnmi package (often not in the default image), and the BGP YANG model is still maturing.

SNMP BGP4-MIB is the legacy alternative. The bgpPeerState, bgpPeerEstablishedTransitions, bgpPeerInUpdates, bgpPeerOutUpdates counters provide session-level telemetry. The cost: the BGP4-MIB does not provide full RIB state (only summary counters), and the counters are 32-bit.

The canonical BGP state

The operator must know which BGP state structures to monitor. The four canonical state structures:

flowchart TD
  subgraph BGP_STATE["BGP state"]
    SE["Session state<br/>(Idle, Connect, Active, OpenSent, OpenConfirm, Established)"]
    RT["Route state<br/>(Loc-RIB, Adj-RIBs-In, Adj-RIBs-Out)"]
    UP["Update state<br/>(sent, received, queued)"]
    AT["Attribute state<br/>(origin, AS-path, MED, local-pref, community)"]
  end
  SE --> RT
  RT --> UP
  UP --> AT

Session state is the BGP FSM state. The operator monitors bgpPeerState (or Established in show ip bgp summary) to know whether the session is up. A session that is bouncing between Active and Idle is failing to establish; a session that is Established but flapping every few minutes is a keepalive or hold-time issue.

Route state is the RIB. The operator monitors bgpLocalAs, bgpLocalAddr, the number of received prefixes, the number of accepted prefixes, and the number of rejected prefixes. The Adj-RIBs-In is the per-peer received RIB; the Loc-RIB is the router’s best-path selection; the Adj-RIBs-Out is what the router advertises to its peers.

Update state is the update message rate. The operator monitors bgpPeerInUpdates (updates received) and bgpPeerOutUpdates (updates sent). A spike in bgpPeerInUpdates indicates a peer is sending a large route refresh; a spike in bgpPeerOutUpdates indicates the router is computing a new best path and advertising it.

Attribute state is the per-prefix attributes. The operator monitors the most common attributes: origin (IGP, EGP, incomplete), as-path (the AS path), med (the multi-exit discriminator), local-pref (the local preference), and community (the communities).

Step 1 — Configure BMP

The operator configures BMP to stream BGP state to a collector. BMP is the canonical production transport for BGP telemetry.

configure
set protocols bmp target 10.0.0.50 port 5000
set protocols bmp policy prefix-list monitor-all
set policy prefix-list monitor-all rule 10 action permit
set policy prefix-list monitor-all rule 10 prefix '0.0.0.0/0'
set policy prefix-list monitor-all rule 10 ge 0
set policy prefix-list monitor-all rule 10 le 32
commit

The target is the BMP collector’s IP and port. The policy prefix-list determines which routes the BMP collector sees: 0.0.0.0/0 with ge 0 le 32 matches every IPv4 prefix. For IPv6, the operator adds a separate prefix-list entry with ::/0 ge 0 le 128.

The operator validates the BMP session:

$ vtysh -c "show bmp summary"
BMP neighbor 10.0.0.50, port 5000
  Established: yes
  Message Count: 1234
  Byte Count: 567890
  Last Keepalive: 00:00:05
  Last Update: 00:00:10

The Established: yes confirms the BMP session is up. The Message Count and Byte Count confirm the collector is receiving data. If Established: no, the operator checks the collector’s reachability, the firewall, and the BMP configuration.

Step 2 — Use FRR’s northbound JSON

The operator uses show ip bgp summary json to inspect the BGP state in a machine-readable format. This is the canonical pattern for ad-hoc inspection and for automation playbooks.

$ vtysh -c "show ip bgp summary json" | jq .
{
  "ipv4Unicast": {
    "peers": {
      "10.0.1.1": {
        "remoteAs": 65001,
        "state": "Established",
        "msgRcvd": 1234567,
        "msgSent": 7654321,
        "upDownTime": "21:00:00",
        "pfxRcd": 123456,
        "pfxSnt": 0
      },
      "10.0.1.2": {
        "remoteAs": 65002,
        "state": "Established",
        "msgRcvd": 2345678,
        "msgSent": 8765432,
        "upDownTime": "21:00:00",
        "pfxRcd": 234567,
        "pfxSnt": 0
      }
    }
  }
}

The output is JSON. The operator uses jq to extract the relevant fields:

$ vtysh -c "show ip bgp summary json" | \
  jq '.ipv4Unicast.peers | to_entries[] | select(.value.state == "Established") | {peer: .key, pfxRcd: .value.pfxRcd, upDownTime: .value.upDownTime}'

For a single peer’s RIB, the operator uses show ip bgp neighbor <peer> received-routes json:

$ vtysh -c "show ip bgp neighbor 10.0.1.1 received-routes json" | jq '.routes | length'
123456

The JSON output is the canonical source for automation. The operator writes a script that alerts if pfxRcd drops by more than 10% (a peer lost routes) or if state is not Established (the session is down).

Step 3 — Stream BGP state via gNMI

The operator enables the gNMI server and subscribes to the BGP YANG model:

configure
set service gnmi listen-address '10.0.0.1'
set service gnmi port '57400'
set service gnmi authentication username 'gnmi-user'
set service gnmi authentication password 'GNm1-2026-Q3'
commit

The operator then uses a gNMI client to subscribe to the BGP state:

$ gnmic -a 10.0.0.1:57400 -u gnmi-user -p 'GNm1-2026-Q3' \
  subscribe --path '/network-instances/network-instance[name=default]/protocols/protocol[identifier=BGP]/bgp' \
  --stream-mode sample --sample-interval 30s

The stream produces BGP state updates every 30 seconds. The operator feeds the stream into a time-series database. The cost: gNMI requires the vyos-gnmi package, and the BGP YANG model is still maturing (the model may not include every field the operator needs).

Step 4 — Configure the Prometheus exporter

The bgp_exporter (from the prometheus-bgp-exporter package) translates the FRR JSON output into Prometheus metrics. The exporter runs alongside the node_exporter and exposes a /metrics endpoint.

configure
set service prometheus node-exporter listen-address '10.0.0.1'
set service prometheus node-exporter port '9100'
set service prometheus node-exporter collectors 'network'
set service prometheus bgp-exporter listen-address '10.0.0.1'
set service prometheus bgp-exporter port '9101'
commit

The operator curls the BGP exporter’s endpoint:

$ curl -s http://10.0.0.1:9101/metrics | grep bgp_
bgp_up{peer="10.0.1.1"} 1
bgp_up{peer="10.0.1.2"} 1
bgp_prefix_count{peer="10.0.1.1",address_family="ipv4_unicast"} 123456
bgp_prefix_count{peer="10.0.1.2",address_family="ipv4_unicast"} 234567
bgp_peer_count{state="Established"} 2

The exporter translates the FRR JSON output into Prometheus metrics. The Prometheus server records these metrics and the operator can alert on bgp_up == 0 (the session is down) or bgp_prefix_count dropping by more than 10% (the peer lost routes).

Validation discipline

Every BGP telemetry metric must be validated before it becomes an alert. The validation discipline:

1. Confirm the BMP session is up. The operator runs show bmp summary and confirms Established: yes.
2. Confirm the FRR JSON output is parseable. The operator runs show ip bgp summary json | jq . and confirms the JSON is valid.
3. Confirm the gNMI subscription is receiving updates. The operator runs the gNMI client and confirms updates are arriving at the expected cadence.
4. Confirm the Prometheus exporter is exposing metrics. The operator curls the exporter and confirms the metrics are present.
5. Confirm the metric is changing. A metric that is present but constant is stale. The operator reads the metric twice, 5 seconds apart, and confirms the counter has incremented.
6. Confirm the alert fires under the expected condition. The operator tests the alert by stopping a BGP session and confirming the NMS alerts.

Production failure modes

The BGP telemetry failure modes the operator encounters:

• BMP collector undersized. The collector cannot keep up with the RIB at session establishment; messages are dropped; the operator sees gaps in the telemetry. Fix: size the collector for the full RIB (RAM + disk).
• BMP session flaps. The BMP session is established but flaps every few minutes; the collector sees an Init message every few minutes and the RIB is re-initialised. Fix: check the collector’s reachability, the BMP keepalive, and the firewall.
• FRR JSON output is truncated. The show ip bgp summary json output is truncated for very large RIBs. Fix: use show bgp ipv4 unicast count json for a summary or filter the output.
• gNMI subscription overruns the collector. A 1-second sample interval on 10,000 BGP paths produces 10,000 updates per second. The collector cannot keep up. Fix: increase the sample interval or use on-change subscriptions.
• Prometheus exporter metrics are stale. The exporter caches the FRR JSON output for 30 seconds; a BGP incident that resolves in 10 seconds is invisible. Fix: reduce the cache interval or use the FRR JSON directly.
• Telemetry flooded with update bursts. A peer sends 100,000 routes in a route refresh; the BMP collector sees a 100,000-message burst; the collector’s queue overflows. Fix: rate-limit the peer (BGP neighbor route-map to bound the prefix count) and add buffering to the collector.

Rollback

BGP telemetry changes are typically configuration-only, but the impact can be cross-cutting. The rollback discipline:

• BMP, FRR JSON, gNMI, Prometheus exporter — all are in the VyOS tree. The rollback is rollback N and commit.
• The bgp_exporter binary — installed outside the VyOS tree. The rollback is to uninstall the package or kill the process.
• BMP collector configuration — the collector is a separate system. The rollback is to revert the collector’s configuration, not the router.

For every change, use commit-confirm:

configure
# ... make the change ...
commit-confirm 5
# If the change has unintended consequences, the auto-rollback
# fires after 5 minutes and the previous configuration is restored.

Production discipline

Cross-course references

• Part XXIV (vyos-xxiv-01-bgp-config) covers the BGP configuration that the telemetry layer is built on.
• Part XXIII (vyos-xxiii-04-bgp-rib) covers the BGP RIB structure that the telemetry layer exposes.
• Part XXXI (vyos-xxxi-01-session-states) covers the BGP session states that the telemetry layer monitors.
• The Observability course covers the consumer side: Prometheus, Grafana, alerting on BGP state deviation.
• The Ansible course’s XLII-Ansible-BeyondLinux covers the automation hand-off (rolling out BMP to a fleet via a single playbook).
• The Linux course’s XXII-Linux-NetTroubleshoot covers the underlying TCP session primitives that BMP uses.

Quiz