Trust boundaries — trust at access, untrusted at WAN, the operator's discipline

A QoS trust boundary is the point in the network where DSCP markings are trusted or untrusted. At the access (LAN), the operator trusts the markings (because the operator controls the LAN); at the WAN, the markings are untrusted (because external hosts could mark anything). The marking strategy determines where the boundary is and what the operator does at that boundary.

This lesson covers the discipline of trust boundaries: trust at access, untrust at WAN, re-mark at the boundary, and the production patterns.

What is a trust boundary

The operator marks some traffic with high priority (e.g., voice with DSCP EF). The QoS policy honours the marks and gives the marked traffic priority. This is useful — but only if the marking is trustworthy.

If any host on the network can mark its own DSCP, the trust is broken. A host could mark its packets as voice (DSCP EF) and get priority treatment for its bulk traffic. This defeats the purpose of QoS and starves legitimate voice traffic.

flowchart LR
  TRUST["TRUSTED: Access (LAN)"]
  UNTRUST["UNTRUSTED: WAN"]
  HOST1["Host marks own DSCP<br/>(untrusted)"]
  HOST2["Operator marks at router<br/>(trusted)"]
  TRUST --> HOST1
  TRUST --> HOST2
  HOST1 -. "should not be marked" .-> UNTRUST
  HOST2 -- "marked with operator's policy" --> UNTRUST
  UNTRUST -. "external markings untrusted" .-> TRUST

The trust boundary is the point where:

• Trusted side (access, LAN, operator-controlled): DSCP markings are trusted. QoS policy honours them.
• Untrusted side (WAN, external): DSCP markings are untrusted. The operator re-marks them to BE (or applies per-traffic marking).

The boundary is typically the router/firewall between the LAN and the WAN.

The marking strategy

A typical marking strategy:

1. Hosts do not mark. Host software does not set DSCP. The router controls marking.
2. The operator’s router marks. The router inspects packets at the LAN boundary (e.g., via DPI or port-based classification) and marks them with the appropriate DSCP.
3. The WAN interface re-marks. At the WAN boundary, the router re-marks all unmarked traffic to BE (DSCP 0). Marked traffic from external sources is also re-marked to BE (because external markings are untrusted).
4. Internal network honours DSCP. Internal routers honour the DSCP markings; voice traffic gets priority.

# Host on the LAN sends traffic
# (no DSCP marking; default BE)
#
# Router at LAN boundary classifies and marks
# (e.g., VoIP → EF, video → AF41, bulk → BE)
set firewall ipv4 mangle rule 10 action accept
set firewall ipv4 mangle rule 10 protocol udp
set firewall ipv4 mangle rule 10 destination port 5060
set firewall ipv4 mangle rule 10 set dscp 46
# (Now packets are EF-marked at the LAN egress)
#
# Internal routers honour DSCP
set qos policy shaper LAN-OUT class voice match dscp 46
# (Voice class honours EF)
#
# WAN interface re-marks external markings to BE
set firewall ipv4 mangle rule 100 action accept
set firewall ipv4 mangle rule 100 protocol ip
set firewall ipv4 mangle rule 100 set dscp 0
# (Re-mark all traffic at the WAN boundary)

The strategy:

• LAN: trust the operator’s markings.
• Internal: honour DSCP.
• WAN: strip external markings; re-mark to BE.

Configure trust boundaries on VyOS

A typical trust boundary configuration:

configure
# LAN interface (trust)
set interfaces ethernet eth1 address 192.168.1.1/24
set interfaces ethernet eth1 traffic-policy out LAN-OUT
# QoS policy on the LAN honours DSCP

# WAN interface (untrust)
set interfaces ethernet eth0 address 198.51.100.1/24
set interfaces ethernet eth0 traffic-policy out WAN-OUT
# QoS policy on the WAN does NOT honour DSCP

# Marking rules on the LAN
set firewall ipv4 mangle rule 10 action accept
set firewall ipv4 mangle rule 10 protocol udp
set firewall ipv4 mangle rule 10 destination port 5060
set firewall ipv4 mangle rule 10 set dscp 46

set firewall ipv4 mangle rule 20 action accept
set firewall ipv4 mangle rule 20 protocol tcp
set firewall ipv4 mangle rule 20 destination port 1720
set firewall ipv4 mangle rule 20 set dscp 34

# Re-marking on the WAN (strip external markings)
set firewall ipv4 mangle rule 100 action accept
set firewall ipv4 mangle rule 100 chain postrouting
set firewall ipv4 mangle rule 100 out-interface eth0
set firewall ipv4 mangle rule 100 set dscp 0

commit
save

The configuration:

• LAN-facing interface has markings; QoS policy honours DSCP.
• WAN-facing interface has re-marking (to BE); QoS policy does NOT honour DSCP.

The marking rules apply at the LAN boundary; the re-marking rules apply at the WAN boundary.

Trust boundaries in multi-tenant networks

In a multi-tenant network (e.g., a service-provider network or a data center with many tenants), the trust boundary is more nuanced:

• Provider’s core: trust the operator’s markings.
• Tenant’s access: the tenant marks their own traffic (within the tenant’s contract).
• Tenant-to-tenant boundary: re-mark at the boundary; each tenant’s markings are scoped to the tenant.

A typical multi-tenant approach:

• The provider’s edge router marks the tenant’s traffic with a per-tenant DSCP (e.g., tenant A → EF, tenant B → AF41).
• The provider’s core honours the per-tenant DSCP.
• The provider’s egress re-marks to BE (preventing external influence).

flowchart LR
  TENANT_A["Tenant A's traffic"]
  TENANT_B["Tenant B's traffic"]
  EDGE["Provider edge (mark)"]
  CORE["Provider core (honour DSCP)"]
  EGRESS["Provider egress (re-mark to BE)"]
  TENANT_A --> EDGE
  TENANT_B --> EDGE
  EDGE --> CORE
  CORE --> EGRESS
  EGRESS --> EXT["External"]

The trust boundary is at the provider’s egress — external markings are stripped.

Validation

# Verify the marking rules on the LAN
iptables -t mangle -L -v
# Lists the marking rules with packet counts

# Verify the re-marking rules on the WAN
iptables -t mangle -L -v
# Should show the re-marking rule on the WAN chain

# Test with captures
# Capture on the LAN (eth1)
tcpdump -ni eth1 -c 10 -vv
# Verify the DSCP on the packets: EF (46) for VoIP, AF41 (34) for video, BE (0) for default

# Capture on the WAN (eth0)
tcpdump -ni eth0 -c 10 -vv
# Verify the DSCP on the packets: BE (0) for all traffic (re-marked)

A clean validation: markings are applied at the LAN; re-markings are applied at the WAN; the DSCP is consistent with the trust policy.

Production failure modes

Hosts marking their own DSCP

Hosts mark their own DSCP, bypassing the trust boundary. The QoS policy honours the marking, and the hosts get priority.

Diagnostic: tcpdump on the LAN shows packets with high DSCP values that are not from the router’s marking rules.

Fix: configure the router to re-mark all unmarked traffic to BE on the LAN ingress; only the router’s own marking rules apply.

Re-marking on the wrong interface

The re-marking rules are on the LAN interface instead of the WAN interface. External markings are still honoured on the WAN egress.

Fix: move the re-marking rules to the WAN interface (postrouting chain, out-interface = WAN).

No re-marking at WAN

The WAN interface does not have re-marking rules. External markings are honoured (against the policy).

Fix: add the re-marking rules on the WAN interface (POSTROUTING chain, out-interface = WAN).

Over-marking by the operator

The operator marks all traffic as EF. The QoS policy cannot differentiate. The marking loses meaning.

Fix: review the marking rules; ensure each class has a meaningful DSCP value.

Rollback

# Enter configuration mode and write the running configuration to a
# file you can load back. `save` is a configuration-mode command that
# takes a path; operational mode has no `| save` pipe.
configure
save /config/pre-change-qos-trust-TICKET.conf

# Remove the trust boundary configuration
delete firewall ipv4 mangle rule 100
delete qos policy shaper LAN-OUT
delete interfaces ethernet eth1 traffic-policy out LAN-OUT

# Read the diff before committing anything
compare
commit

# Or restore a previous configuration
load /config/pre-change-qos-trust-TICKET.conf
commit
save

The rollback removes the trust boundary; external markings are honoured (which is a security risk).

Production discipline

Cross-course references

• Part XLV-01 (XLV-VyOS-QoS / concept) covers the overall QoS pipeline.
• Part XLV-02 (XLV-VyOS-QoS / DSCP marking) covers DSCP marking, a key part of trust boundaries.
• Part XXXVII-05 (XXXVII-VyOS-Firewall / default-deny) covers the firewall rules that the trust boundary depends on.

Quiz