MetalLB BGP mode — multi-node load balancing with ECMP

MetalLB BGP mode advertises Service IPs via BGP, allowing multi-node load balancing with ECMP and fast failover. This lesson walks the mechanism, the configuration, BGP communities, and the discipline.

How BGP mode works

flowchart LR
    A[Service LoadBalancer] --> B[MetalLB Controller]
    B --> C["IPAddressPool: allocate IP"]
    C --> D["Speaker: every node"]
    D -->|BGP advertise| E[Router]
    E -->|ECMP| F[Node 1]
    E -->|ECMP| G[Node 2]
    E -->|ECMP| H[Node 3]
    F -->|kube-proxy| I[Service backend]
    G --> I
    H --> I

The BGP mechanism:

1. The MetalLB controller allocates an IP from the IPAddressPool.
2. Every Speaker (one per node) advertises the IP via BGP to the upstream router.
3. The router sees multiple equal-cost paths (ECMP) and load-balances across them.
4. Each node’s kube-proxy distributes traffic to Service backends.
5. If a node fails, BGP withdraws its advertisement; the router stops sending traffic to it. BFD (Bidirectional Forwarding Detection) detects the failure in < 1 second.

The configuration

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: prod-pool
  namespace: metallb-system
spec:
  addresses:
    - 192.168.100.0/24
---
apiVersion: metallb.io/v1beta2
kind: BGPPeer
metadata:
  name: prod-router
  namespace: metallb-system
spec:
  peerAddress: 192.168.1.1
  peerASN: 64512
  myASN: 64513
  holdTime: 90s
  keepaliveTime: 30s
---
apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
  name: prod-advert
  namespace: metallb-system
spec:
  ipAddressPools:
    - prod-pool
  peers:
    - prod-router
  communities:
    - 64512:100

The configuration:

• IPAddressPool. The range of IPs.
• BGPPeer. The router’s IP, ASN; MetalLB’s ASN; timers.
• BGPAdvertisement. Binds the pool to the peer; BGP communities.

# On the router (Cisco IOS example)
show ip bgp neighbors

Verify the BGP sessions are established and the IPs are advertised.

BGP communities

flowchart LR
    A[MetalLB advertises IP] --> B["With community 64512:100"]
    B --> C[Router receives]
    C --> D{Match community?}
    D -->|64512:100| E[Set local-preference 200]
    D -->|Other| F[Default preference]
    E --> G[Route preferred]

BGP communities are tags attached to BGP routes. The router can match on communities and apply routing policy:

• 64512:100 — tag MetalLB-originated routes for the prod pool.
• 64512:200 — tag for a different pool; different local-preference.

Communities allow granular routing policy without manual filter lists.

BFD for fast failover

apiVersion: metallb.io/v1beta2
kind: BGPPeer
metadata:
  name: prod-router
spec:
  peerAddress: 192.168.1.1
  peerASN: 64512
  myASN: 64513
  bfd:
    receiveInterval: 50ms
    transmitInterval: 50ms
    multiplier: 3

BFD (Bidirectional Forwarding Detection) detects peer failure in milliseconds:

• receiveInterval / transmitInterval: how often BFD packets are sent.
• multiplier: how many missed packets before declaring failure (3 × 50ms = 150ms total).

With BFD, failover is sub-second instead of 30-60 seconds (default BGP timers).

The failover

flowchart LR
    A[Node fails] --> B[BFD detects in milliseconds]
    B --> C[BGP withdraws the route]
    C --> D[Router stops sending traffic]
    D --> E[Other nodes continue serving]
    E --> F[Sub-second failover]

The failover:

1. A node fails (network, power, etc.).
2. BFD detects the failure in milliseconds.
3. The Speaker on the failed node withdraws its BGP advertisement.
4. The router stops sending traffic to the failed node.
5. The remaining nodes continue serving traffic.
6. End-to-end failover: < 1 second.

The operational trade-offs

flowchart LR
    A[BGP mode] --> B[+ Multi-node load balancing]
    A --> C[+ ECMP for throughput]
    A --> D[+ Sub-second failover with BFD]
    A --> E[- Requires router config]
    A --> F["- AS numbers, communities"]
    A --> G[- Operational complexity]

The trade-offs:

• Pros. Multi-node; ECMP; sub-second failover.
• Cons. Requires router config; AS numbers, communities; operational complexity.

Quiz

The operational discipline

MetalLB BGP in production rests on five non-negotiable elements:

• BGP communities. Use them for routing policy.
• BFD for fast failover. Sub-second failover is essential for production.
• Verify BGP sessions. Use show ip bgp summary on the router.
• Test failover. Disconnect a node; verify traffic redistributes.
• Document the topology. IP pools, peers, ASNs, communities, BFD configuration.

MetalLB BGP is the production-grade load balancer for on-prem. The discipline is to test failover, monitor BGP sessions, and document the configuration.