Queuing and shaping — HTB tree, fq_codel leaves, bandwidth shaping

HTB (Hierarchical Token Bucket) and fq_codel are the typical Linux QoS components for production deployments. HTB provides the tree structure (classes, rates, ceilings) for shaping; fq_codel provides fair queuing and low-latency at the leaf. The combination gives both shaping (rate limits per class) and fairness (low latency under load).

This lesson covers HTB configuration (rate, ceiling, burst), fq_codel leaves, bandwidth shaping on VyOS, and the production failure modes.

HTB (Hierarchical Token Bucket)

HTB is a classful qdisc that provides:

• Hierarchical structure: classes can have child classes, forming a tree.
• Rate guarantees: each class has a rate (the guaranteed minimum bandwidth).
• Ceiling (ceiling): each class has a ceil (the maximum bandwidth; classes can use more bandwidth when available).
• Burst: each class has a burst (the amount of bytes that can be sent in a burst).

flowchart LR
  ROOT["Root (rate = link capacity)"]
  CLASS_A["Class A (voice)<br/>rate = 100 Mbps<br/>ceil = 200 Mbps"]
  CLASS_B["Class B (video)<br/>rate = 200 Mbps<br/>ceil = 300 Mbps"]
  CLASS_C["Class C (bulk)<br/>rate = 500 Mbps<br/>ceil = 1 Gbps"]
  ROOT --> CLASS_A
  ROOT --> CLASS_B
  ROOT --> CLASS_C

A typical HTB tree:

• Root class with rate = link capacity (e.g., 1 Gbps).
• Class A (voice): guaranteed 100 Mbps, can use up to 200 Mbps when available.
• Class B (video): guaranteed 200 Mbps, can use up to 300 Mbps.
• Class C (bulk): guaranteed 500 Mbps, can use up to 1 Gbps.

When the link is fully utilised, each class gets its rate. When the link has spare capacity, classes can use up to their ceil (in priority order; higher-priority classes get spare capacity first).

fq_codel (Fair Queuing Controlled Delay)

fq_codel is the modern leaf scheduler. It combines:

• Fair queuing: each flow gets its own queue; flows are served in a round-robin manner.
• Controlled delay (CoDel): a queue management algorithm that drops packets from queues with excessive delay (per RFC 8289).

fq_codel provides:

• Low latency under load: CoDel targets 5 ms delay; packets in queues exceeding the target get dropped to maintain the target.
• Fairness: each flow gets equal share of the leaf scheduler.
• No configuration needed: the default parameters are good (target = 5 ms, interval = 100 ms).

# Use fq_codel as the leaf scheduler
set qos policy shaper WAN-OUT default queue-type fq-codel
set qos policy shaper WAN-OUT class voice queue-type fq-codel
set qos policy shaper WAN-OUT class video queue-type fq-codel
set qos policy shaper WAN-OUT class bulk queue-type fq-codel

fq_codel is the recommended leaf scheduler for most deployments. It handles transient congestion gracefully, doesn’t starve flows, and provides low latency.

Configure bandwidth shaping on VyOS

A full HTB + fq_codel configuration:

configure
# Root with 1 Gbps link
set qos policy shaper WAN-OUT default bandwidth 1gbit
set qos policy shaper WAN-OUT default burst 15k
set qos policy shaper WAN-OUT default queue-type fq-codel

# Voice class
set qos policy shaper WAN-OUT class voice match dscp 46
set qos policy shaper WAN-OUT class voice priority 0
set qos policy shaper WAN-OUT class voice bandwidth 100mbit
set qos policy shaper WAN-OUT class voice burst 15k
set qos policy shaper WAN-OUT class voice queue-type fq-codel
set qos policy shaper WAN-OUT class voice ceiling 200mbit

# Video class
set qos policy shaper WAN-OUT class video match dscp 34
set qos policy shaper WAN-OUT class video priority 2
set qos policy shaper WAN-OUT class video bandwidth 200mbit
set qos policy shaper WAN-OUT class video burst 15k
set qos policy shaper WAN-OUT class video queue-type fq-codel
set qos policy shaper WAN-OUT class video ceiling 300mbit

# Bulk class (default)
set qos policy shaper WAN-OUT class bulk match all
set qos policy shaper WAN-OUT class bulk priority 7
set qos policy shaper WAN-OUT class bulk bandwidth 500mbit
set qos policy shaper WAN-OUT class bulk burst 15k
set qos policy shaper WAN-OUT class bulk queue-type fq-codel
set qos policy shaper WAN-OUT class bulk ceiling 1gbit

# Apply to WAN egress
set interfaces ethernet eth0 traffic-policy out WAN-OUT

commit
save

The configuration:

• Bandwidth / ceiling: rate guarantees and maximum per class.
• Burst: amount of bytes that can be sent at line rate (before rate-limited).
• Queue-type: fq_codel as the leaf.
• Priority: lower number = higher priority (0 is highest).

Validation

# Verify the QoS policy
show qos
# Lists the policy and classes

# Detailed class statistics
tc -s class show dev eth0
# Lists the classes with packet counts and drops

# Detailed queue statistics
tc -s qdisc show dev eth0
# Lists the qdiscs with statistics

# Test with iPerf3
iperf3 -c <server> -P 4
# Should achieve close to the bandwidth limit

# Verify with multiple classes
# Generate traffic matching different classes; observe the counters
# (Hard to set up exactly; use captures to identify the traffic)

A clean validation: the classes are configured; the bandwidth limits are applied; the leaf scheduler (fq_codel) is in effect; iPerf3 shows expected throughput per class.

Production failure modes

Ceiling too low

The voice class has ceil 100 Mbps but needs more during a busy hour. The class drops excess packets.

Diagnostic: tc -s class show dev eth0 shows drops in the voice class.

Fix: increase the ceiling.

Rate too high

The sum of class rates exceeds the link capacity. The router cannot guarantee the rates; under congestion, some classes do not get their rate.

Fix: ensure the sum of class rates ≤ link capacity.

Queue-type not fq_codel

The leaf scheduler is SFQ or FIFO (the legacy defaults). The latency is not controlled.

Fix: change the queue-type to fq-codel.

Burst too low

The burst is too small; short spikes are rate-limited.

Fix: increase the burst.

Queue build-up without drops

The class queue grows but drops are rare. The queue provides buffering; under sustained overload, the queue grows unboundedly until the kernel’s default limit (or the operator’s explicit limit) is reached.

Diagnostic: tc -s class show dev eth0 shows increasing qlen but no drops.

Fix: increase the queue limit (or add a policer to drop excess).

Rollback

# Enter configuration mode and write the running configuration to a
# file you can load back. `save` is a configuration-mode command that
# takes a path; operational mode has no `| save` pipe.
configure
save /config/pre-change-qos-shape-TICKET.conf

# Remove the QoS shaping
delete qos policy shaper WAN-OUT
delete interfaces ethernet eth0 traffic-policy out WAN-OUT

# Read the diff before committing anything
compare
commit

# Or restore a previous configuration
load /config/pre-change-qos-shape-TICKET.conf
commit
save

The rollback removes the QoS policy; the interface reverts to FIFO.

Production discipline

Cross-course references

• Part XLV-01 (XLV-VyOS-QoS / concept) covers the overall QoS pipeline.
• Part XLV-02 (XLV-VyOS-QoS / DSCP marking) covers DSCP marking that HTB may use for classification.
• Part XLV-05 (XLV-VyOS-QoS / trust boundaries) covers the trust-boundary discipline.

Quiz