Lab: Configure source-based policy routing for two uplinks

This lab builds source-based policy routing on a host with two uplinks. Replies to traffic from uplink A go out uplink A; replies to traffic from uplink B go out uplink B.

Objective

By the end of this lab, you can:

• Add a custom routing table by name and number.
• Add routes to the custom table.
• Add a rule that sends traffic from a specific source to the custom table.
• Verify routing decisions with ip route get.
• Recognise when this pattern generalises to multi-homing.

Architecture

The host has two interfaces:

• eth0 (uplink A): address 10.0.0.10/24, gateway 10.0.0.1.
• eth1 (uplink B): address 192.168.1.10/24, gateway 192.168.1.1.

Goals:

• Traffic from 10.0.0.10 (eth0’s source) uses eth0’s default route.
• Traffic from 192.168.1.10 (eth1’s source) uses eth1’s default route.
• All other traffic uses eth0’s default route.

In a real lab, eth0 and eth1 connect to two real uplinks or two VMs acting as uplinks. In this read-only exploration, the ip route get tests will show what the kernel would do.

Tasks

Task 1: Read the current state

ip -br link show
ip -br addr show
ip route show
ip rule show

Write down the current tables and rules. The default is local, main, default.

Task 2: Add a custom table

echo "100 backup" | sudo tee -a /etc/iproute2/rt_tables
cat /etc/iproute2/rt_tables

The number 100 is the table ID; backup is the name.

Task 3: Add routes to the custom table

sudo ip route add 192.168.1.0/24 dev eth1 src 192.168.1.10 table backup
sudo ip route add default via 192.168.1.1 dev eth1 table backup

Verify:

ip route show table backup

Task 4: Add a rule

sudo ip rule add from 192.168.1.10 lookup backup priority 100
ip rule show

Verify the rule is present.

Task 5: Test routing decisions

ip route get 8.8.8.8 from 10.0.0.10
ip route get 8.8.8.8 from 192.168.1.10
ip route get 8.8.8.8

The first query should pick eth0. The second should pick eth1 (via table backup). The third (no source) uses the main table - eth0.

Task 6: Test end-to-end (if uplinks are real)

ping -I eth0 8.8.8.8
ping -I eth1 8.8.8.8

-I binds the source to the specified interface. With real uplinks, both should succeed. With their replies going out the same interface they came in on (no asymmetric routing).

Task 7: Test what happens if the custom table has no route

sudo ip route add blackhole 8.8.8.8 table backup
ip route get 8.8.8.8 from 192.168.1.10

The query should return blackhole. The custom table is consulted first (priority 100) and the blackhole route wins.

Remove the blackhole to restore normal behaviour:

sudo ip route del blackhole 8.8.8.8 table backup

Task 8: Test with ip route flush cache

sudo ip route flush cache
ip route get 8.8.8.8 from 192.168.1.10

The routing cache may serve stale decisions; flushing forces re-evaluation.

Task 9: Persist with Netplan

# /etc/netplan/99-policy-routing.yaml
network:
  version: 2
  renderer: networkd
  ethernets:
    eth0:
      addresses: [10.0.0.10/24]
      routes:
        - to: default
          via: 10.0.0.1
    eth1:
      addresses: [192.168.1.10/24]
      dhcp4: no
  routing:
    policy:
      - from: 192.168.1.10
        table: 100
        priority: 100
    tables:
      - id: 100
        name: backup
        routes:
          - to: 192.168.1.0/24
            dev: eth1
          - to: default
            via: 192.168.1.1
            dev: eth1

Apply safely:

sudo netplan try

Task 10: Document the rollback

sudo rm /etc/netplan/99-policy-routing.yaml
sudo netplan apply

This restores the earlier configuration.

Validation

• A custom table 100 backup exists with routes for eth1’s subnet and default.
• A rule with priority 100 sends traffic from 192.168.1.10 to that table.
• ip route get 8.8.8.8 from 192.168.1.10 returns eth1’s default route.
• ip route get 8.8.8.8 from 10.0.0.10 returns eth0’s default route.
• The configuration survives a reboot (after netplan apply).

Cleanup

Revert by removing the Netplan file and applying.

What you learned

• Source-based policy routing is the right tool for multi-homing.
• A custom routing table holds routes that are only consulted when a rule sends traffic there.
• ip route get <dest> from <src> is the diagnostic of choice.
• The table ID is what ip rule consults; the name is just a label.