LACP — the protocol that aggregates links

Two cables between a router and a switch are two links. Making them behave as one logical link is a negotiation, and the Link Aggregation Control Protocol is the negotiation. LACP is defined by IEEE 802.1AX (the specification moved there from IEEE 802.3ad in 2008, which is why the Linux bonding driver still calls the mode 802.3ad). This lesson covers what the protocol actually agrees, what it refuses to agree, and how VyOS 1.5 LTS exposes it.

The rest of Part X depends on this one. Mode selection (vyos-x-02-bond-modes) is a choice about whether you want this negotiation at all; switch dependencies (vyos-x-03-switch-dependencies) is a statement of what the far end must be configured to do to complete it.

What aggregation buys, and what it does not

A bundle of two 1 Gbps links is not a 2 Gbps link. It is two 1 Gbps links that a hash function distributes traffic across. That distinction is the source of most disappointed expectations in production:

Aggregate throughput rises. Many conversations, spread across members, can collectively exceed one member’s capacity.
Single-flow throughput does not. A hash pins one conversation to one member for the life of that conversation, so a single TCP stream is still limited to one member’s line rate.
Failure of one member is survivable. The remaining members carry the traffic, at reduced aggregate capacity.

LACP negotiates membership of the bundle. It does not negotiate the hash, and it does not police capacity. Each end chooses its own transmit hash independently, which is why a bundle can be perfectly aggregated and still be unevenly loaded.

The LACP conversation

Each end of each link sends a LACP Data Unit (LACPDU) describing itself — the actor — and describing what it last heard from the far end — the partner. When the actor’s view of the partner matches what the partner says about itself, the two ends are synchronised and the link can join the bundle.

Three fields decide which links may aggregate together:

Field	What it is	Why it matters
System ID	System priority plus the system MAC address	Identifies the device. Links whose partners report different system IDs cannot join one bundle.
Operational key	A value the implementation derives per port, from properties such as speed and duplex	Links must share a key to aggregate. A 1 Gbps and a 10 Gbps port therefore do not aggregate together.
Port ID	Port priority plus port number	Breaks ties and orders the ports within the bundle.

sequenceDiagram
  participant R as VyOS bond0 (actor)
  participant S as Switch LAG (partner)
  R->>S: LACPDU: actor sys-id A, key 9, port 1<br/>partner unknown
  S->>R: LACPDU: actor sys-id B, key 33, port 1<br/>partner sys-id A, key 9
  R->>S: LACPDU: actor sys-id A, key 9<br/>partner sys-id B, key 33
  Note over R,S: Both ends agree on the partner.<br/>Sync bit set, port collects and distributes.

Until that agreement is reached the link is up at the physical layer and carries no bundle traffic. “The cable is plugged in” and “the link is a member of the aggregation” are different states, and the gap between them is where bonding incidents live.

Active and passive

An LACP participant is either active or passive. An active participant sends LACPDUs unprompted. A passive participant answers LACPDUs but never speaks first.

The consequence is a rule worth memorising: at least one end must be active. Two passive ends sit silently forever, both links up, neither aggregated, and no log line saying why. VyOS drives the bond through the Linux bonding driver, whose 802.3ad implementation transmits LACPDUs, so the far end may be configured either way — but a switch-to-switch or switch-to-firewall bundle elsewhere in the estate is entirely capable of being passive on both ends.

Fast and slow rate

LACPDUs are exchanged periodically. The rate is one of two values:

Rate	LACPDU interval	Timeout (three missed LACPDUs)
`slow`	30 seconds	90 seconds
`fast`	1 second	3 seconds

slow is the driver default. The rate is a request: setting lacp-rate fast asks the partner to transmit every second, and it is the partner’s transmissions that this end times out on. The two ends may run different rates.

LACP versus a static LAG

A static LAG (balance-xor on the host, a channel group with no protocol on the switch) skips the negotiation. Both ends simply assume the bundle exists.

flowchart TD
  A[Two links, one bundle intended] --> B{Protocol running?}
  B -- No: static LAG --> C[Both ends assume membership]
  C --> D[Miswired link is still used<br/>Frames leave into the wrong device]
  B -- Yes: LACP --> E[Partner system ID compared]
  E --> F[Mismatched partner is not aggregated<br/>Link stays out of the bundle]

That is the argument for the protocol. A patching error that lands one member on a different switch is invisible to a static LAG, which keeps hashing frames onto a link that does not reach the intended peer. LACP compares the partner system ID, finds two different devices, and refuses to bundle the odd link — the bundle degrades to the members that agree, which is a survivable outcome with an obvious diagnostic.

Use a static LAG only when the far end genuinely cannot run LACP.

Configuration in VyOS 1.5 LTS

[edit]
vyos@vyos# set interfaces bonding bond0 mode '802.3ad'
[edit]
vyos@vyos# set interfaces bonding bond0 lacp-rate 'fast'
[edit]
vyos@vyos# set interfaces bonding bond0 hash-policy 'layer3+4'
[edit]
vyos@vyos# set interfaces bonding bond0 min-links '1'
[edit]
vyos@vyos# set interfaces bonding bond0 member interface 'eth1'
[edit]
vyos@vyos# set interfaces bonding bond0 member interface 'eth2'
[edit]
vyos@vyos# set interfaces bonding bond0 address '192.0.2.1/24'
[edit]
vyos@vyos# set interfaces bonding bond0 description 'uplink to core LAG'
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# save

The nodes that matter for the negotiation:

mode '802.3ad' — the only mode that runs LACP. Every other mode either needs no far-end configuration or needs a static LAG.
lacp-rate — slow or fast, as above.
hash-policy — layer2, layer2+3 or layer3+4. This is a local transmit decision, not part of the negotiation.
min-links — the number of members that must be aggregated before the bond asserts carrier. With min-links 1 a bond that loses every member goes down rather than accepting traffic it cannot forward.
system-mac — overrides the MAC used in the 802.3ad system ID. Only needed where two devices must present one system ID.

Member interfaces must not carry their own addresses; they belong to the bond, and the bond carries the address.

Validation

The VyOS-level view:

vyos@vyos:~$ show interfaces bonding
vyos@vyos:~$ show interfaces bonding bond0
vyos@vyos:~$ show interfaces bonding bond0 detail

show interfaces bonding summarises every bond; adding the interface name reports that bond’s statistics; detail reports the physical member links.

The kernel driver state is the source of truth, because it exposes the negotiation rather than a summary of it:

vyos@vyos:~$ cat /proc/net/bonding/bond0
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up

802.3ad info
LACP active: on
LACP rate: fast
Min links: 1
System MAC address: 00:53:00:aa:bb:cc
Active Aggregator Info:
	Number of ports: 2
	Partner Mac Address: 00:53:00:11:22:33

Slave Interface: eth1
MII Status: up
Speed: 1000 Mbps
Duplex: full
details actor lacp pdu:
    port state: 63
details partner lacp pdu:
    system mac address: 00:53:00:11:22:33
    port state: 63

Three lines carry the verdict:

Partner Mac Address. All zeroes means no LACPDU has been received on this bundle at all — the far end is not running LACP, or is passive while this end is also passive.
Number of ports. Fewer than the configured members means some member failed to aggregate; compare each member’s partner system MAC to find the odd one out.
port state on both actor and partner. Anything without the synchronisation, collecting and distributing bits is a member that is not carrying traffic.

How it fails

Switch ports are not in a LAG. LACPDUs go out, nothing comes back, partner MAC stays all zeroes. Both links are up and only one is usable, or the switch learns the same MAC on two ports and starts flapping it.
Both ends passive. No LACPDU is ever transmitted. The symptom is identical to the previous one; the difference is that the far end insists it is configured for LACP.
Members at different speeds. 802.1AX requires the members of an aggregation to run at the same speed and duplex; the driver derives different operational keys for them and only the matching subset aggregates.
A member patched to the wrong switch. LACP reports two different partner system IDs and excludes the odd member. On a static LAG the same miswire silently black-holes a share of the traffic.
min-links left at its default. The bond keeps carrier with zero aggregated members, so routing keeps a next hop pointed at an interface that cannot forward.
Expecting a single flow to exceed one member. The bundle is correct; the expectation is not. Measure with parallel streams.

Rollback

Wrong mode or rate: reset the node and commit — set interfaces bonding bond0 lacp-rate 'slow', commit.
Bundle never aggregates and the switch cannot be changed in the window: set interfaces bonding bond0 mode 'active-backup', commit. Redundancy is preserved, aggregate bandwidth is not.
Change made over the bond itself: use commit-confirm so an unreachable result reverts on its own, and hold an out-of-band path open. Part VI covers the mechanics.
Full revert of the change set: rollback 1 and commit.

Production discipline

Cross-course references

The Linux course’s XIX-Linux-NetFoundations covers the kernel bonding driver from the host side. The Proxmox course’s XXIX-Proxmox-Networking covers bonds beneath hypervisor bridges.