Skip to main content
RunBook Academy

KubernetesCXII · Load Balancing on Bare MetalLoad balancing on bare metal

Bare metal LB anti-patterns — the most common mistakes

Advanced⏱ ~16 minkubectlmetallb

What you'll learn

  • Identify the most common bare metal LB anti-patterns
  • Explain why each anti-pattern is a problem
  • Apply the fixes for each anti-pattern
  • Build the operational discipline of treating bare metal LB as production networking

Prerequisites

Verified against Kubernetes 1.34.x · kubeadm 1.34.x · kubectl 1.34.x · etcd 3.6.x · CoreDNS 1.11.x · containerd 1.7.x / 2.x · 2026-08-16

Not yet marked complete on this device.

Bare metal LB anti-patterns are the most common mistakes when configuring load balancing on on-prem clusters. This lesson walks the anti-patterns, the fixes, and the operational discipline.

Anti-pattern 1: LoadBalancer Service without controller

flowchart LR
    A["Service: type LoadBalancer"] --> B{Cloud controller?}
    B -->|No| C[Service stays Pending]
    B -->|Yes| D[Cloud LB created]

The intent is “use LoadBalancer type”; the result is a Service stuck in Pending on bare metal.

The fix: install MetalLB (or Cilium BGP, F5 BIG-IP). The controller handles the LoadBalancer type.

Anti-pattern 2: Single-node LB without failover plan

# Anti-pattern
# Single node, no failover

The intent is “simple”; the result is downtime when the node fails.

The fix: multi-node deployment; BFD for fast failover in BGP mode.

Anti-pattern 3: No BGP communities

# Anti-pattern
apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
  name: prod-advert
spec:
  ipAddressPools:
    - prod-pool
  peers:
    - prod-router
  # No communities

The intent is “default communities”; the result is no way to differentiate routes on the router side.

The fix: always use BGP communities to tag routes for routing policy.

Anti-pattern 4: No BFD

# Anti-pattern
apiVersion: metallb.io/v1beta2
kind: BGPPeer
metadata:
  name: prod-router
spec:
  peerAddress: 192.168.1.1
  peerASN: 64512
  myASN: 64513
  # No bfd: section

The intent is “default timers”; the result is 90-second failover (default BGP hold time).

The fix: configure BFD for sub-second failover.

Anti-pattern 5: Mixed L2 and BGP for same IP pool

# Anti-pattern
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: prod-pool
spec:
  addresses:
    - 192.168.100.0/24
---
# L2Advertisement AND BGPAdvertisement for the same pool

The intent is “use both”; the result is unpredictable behaviour (the IP is advertised via both ARP and BGP).

The fix: choose one mode per pool. L2 for small clusters; BGP for production.

Anti-pattern 6: No BGP session monitoring

# Anti-pattern
# No alerts on BGP session down
# No Prometheus rules

The intent is “BGP is reliable”; the result is silent BGP session failures that take down traffic.

The fix: monitor BGP sessions; alert on down or flapping sessions.

Anti-pattern 7: No IP pool boundaries

# Anti-pattern: single pool for everything
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: all-pool
spec:
  addresses:
    - 192.168.100.0/22  # large range

The intent is “one pool for all”; the result is no boundary between production and development.

The fix: separate pools for production and development, with separate communities for routing policy.

Anti-pattern 8: No documentation

# Anti-pattern: BGP configuration exists but is not documented
# No runbook entry
# No comment in YAML

The intent is “the configuration is obvious”; the result is mystery for the next operator.

The fix: document every BGP configuration in the runbook. Each pool, peer, community, BFD setting.

Quiz

Knowledge check · 4 questions

  1. Q1. Why should BGP session state be monitored rather than only Service status?

  2. Q2. A MetalLB Service whose BGP session has dropped still shows an assigned external IP.

  3. Q3. Resolve the inconsistent forwarding caused by announcing one pool through both L2 and BGP.

    Pool 192.168.100.0/24 has both an L2Advertisement and a BGPAdvertisement in metallb-system. Clients on the same VLAN as the nodes reach 192.168.100.20 by ARP and land on node-2, while clients behind the router are ECMP-routed to node-5. A stateful firewall between the two paths starts dropping return traffic for the routed clients, and sessions reset apparently at random.

  4. Q4. Which MetalLB Prometheus metric reports BGP session state, what value means the session is established, and why does alerting on it matter more than alerting on Pod readiness?

Passing score: 75%. Answers are checked in this browser.

The operational discipline

Bare metal LB in production rests on five non-negotiable elements:

  • Install MetalLB or alternative. No LoadBalancer Service without a controller.
  • Multi-node + BFD. Production HA requires both.
  • BGP communities. Tag routes for routing policy.
  • Monitor BGP sessions. Alert on down or flapping.
  • Document every configuration. In the runbook.

Bare metal LB is production networking. The discipline is to install the right controller, test failover, monitor BGP sessions, and document everything.