OSPF cost tuning — auto-cost reference-bandwidth, manual cost, ECMP

OSPF’s path selection is cost-based: SPF computes the lowest-cost path through the LSDB; the lowest-cost path is the one OSPF installs in the FIB. The cost of a path is the sum of the per-interface costs along the path. The operator who controls the cost controls the routing.

This lesson covers the four cost-tuning primitives the operator uses: the auto-cost reference-bandwidth (the calculation that maps link bandwidth to cost), the per-interface manual cost (an override on a specific link), the route-map metric-set (a per-prefix cost on redistributed routes), and the maximum-paths parameter (how many equal-cost paths OSPF installs in the FIB).

Auto-cost and reference-bandwidth

OSPF’s default cost calculation is:

cost = reference-bandwidth / link-bandwidth

The reference-bandwidth defaults to 100 Mbps. The link-bandwidth is the interface’s reported speed. The result is rounded down to an integer (minimum 1).

The problem with the default 100 Mbps reference is that every link faster than 100 Mbps has a cost of 1. A 1 Gbps link and a 10 Gbps link have the same OSPF cost. OSPF cannot distinguish them; SPF treats them as equal.

flowchart LR
  subgraph "Default reference 100 Mbps"
    L1["10 Mbps -> cost 10"]
    L2["100 Mbps -> cost 1"]
    L3["1 Gbps -> cost 1"]
    L4["10 Gbps -> cost 1"]
  end
  subgraph "Reference 10 Gbps"
    R1["10 Mbps -> cost 1000"]
    R2["100 Mbps -> cost 100"]
    R3["1 Gbps -> cost 10"]
    R4["10 Gbps -> cost 1"]
  end

The fix is to raise the reference-bandwidth to match the deployment’s link speeds:

set protocols ospf auto-cost reference-bandwidth 10000
commit
save

10000 is 10 Gbps in Mbps. With this reference, a 10 Gbps link has cost 1, a 1 Gbps link has cost 10, a 100 Mbps link has cost 100. OSPF can now distinguish the links; SPF picks the higher-bandwidth path.

Per-interface manual cost

When the operator wants a specific link to have a specific cost that the auto-cost calculation does not produce, the manual override is the lever:

set protocols ospf interface eth0 cost 50
set protocols ospf interface eth1 cost 5
commit
save

The manual cost overrides the auto-cost for that interface. The use cases:

• Make a slower path preferred. A 100 Mbps backup link should have a higher cost than a 1 Gbps primary. Set the 100 Mbps to 50 and the 1 Gbps to 5; OSPF prefers the 1 Gbps path.
• Force traffic through a specific link. A 10 Gbps core link should be preferred over a 10 Gbps edge link. Set the edge link to 100; the core stays at the auto-cost 1.
• Implement a routing policy via cost. The operator wants traffic for partner X to prefer a specific path. Set the cost on the link to the partner’s network to 5; traffic follows the cost.

flowchart TB
  subgraph "Without manual cost (auto-cost 10 Gbps reference)"
    A1["R1"] -- "10 Gbps cost 1" --> C1["core"]
    C1 -- "10 Gbps cost 1" --> A2["R2"]
    A1 -- "10 Gbps cost 1" --> E1["edge"]
    E1 -- "10 Gbps cost 1" --> A2
    A2 -. "ECMP: both paths have cost 2" .-> A1
  end
  subgraph "With manual cost on edge (cost 100)"
    A1b["R1"] -- "10 Gbps cost 1" --> C2["core"]
    C2 -- "10 Gbps cost 1" --> A2b["R2"]
    A1b -- "10 Gbps cost 1 (auto)" --> E2["edge<br/>cost 100 (manual)"]
    E2 -- "10 Gbps cost 1 (auto)" --> A2b
    A2b -- "prefers core path (cost 2) over edge (cost 101)" --> A1b
  end

Route-map metric-set on redistributed routes

The route-map applied to a redistribution block (covered in lesson xix-04) can set the metric on the resulting Type-5 LSA. This is the lever for traffic-engineering on external routes:

set policy route-map PARTNER-INTO-OSPF rule 10 action permit
set policy route-map PARTNER-INTO-OSPF rule 10 match ip address prefix-list PARTNER
set policy route-map PARTNER-INTO-OSPF rule 10 set metric 50
set policy route-map PARTNER-INTO-OSPF rule 10 set metric-type 1
set protocols ospf redistribute bgp route-map PARTNER-INTO-OSPF
commit
save

Every BGP route that matches the prefix-list is redistributed with metric 50 and metric-type 1. BGP routes that do not match the prefix-list are not redistributed.

sequenceDiagram
  participant BGP as bgpd RIB
  participant RM as route-map PARTNER-INTO-OSPF
  participant ASBR as OSPF ASBR
  participant LSDB as OSPF LSDB
  BGP->>RM: 203.0.113.0/24
  RM->>RM: match PARTNER -> yes
  RM->>RM: set metric 50, metric-type 1
  RM->>ASBR: permit, metric 50, type 1
  ASBR->>LSDB: Type-5 LSA: 203.0.113.0/24 metric 50 type 1
  BGP->>RM: 198.51.100.0/24
  RM->>RM: match PARTNER -> yes
  RM->>ASBR: permit, metric 50, type 1
  ASBR->>LSDB: Type-5 LSA: 198.51.100.0/24 metric 50 type 1
  BGP->>RM: 8.8.8.0/24
  RM->>RM: match PARTNER -> no
  RM-->>BGP: deny (default action)

The route-map metric-set is also useful for differentiating between multiple external sources. BGP into OSPF gets metric 50; static into OSPF gets metric 10; connected into OSPF gets metric 20. The OSPF cost of an external route now reflects the operator’s intent for that source.

Maximum-paths and ECMP

By default, OSPF installs a single best path for each prefix in the FIB. When two paths have equal cost, OSPF installs only one — losing the opportunity for ECMP (equal-cost multi-path) forwarding. The maximum-paths parameter raises the limit:

set protocols ospf maximum-paths 4
commit
save

With maximum-paths 4, OSPF installs up to four equal-cost paths in the FIB. The kernel (and the ASIC, on hardware forwarders) hashes flows across the paths using a configurable algorithm (covered in Part XXXVI on ECMP). Two 10 Gbps links in parallel become an effective 20 Gbps of capacity.

flowchart LR
  subgraph "Without maximum-paths (default 1)"
    R1["R1"] -- "path A cost 5" --> R2["R2"]
    R1 -- "path B cost 5" --> R2
    R2 -. "FIB has only one path" .-> R1
  end
  subgraph "With maximum-paths 2"
    R1b["R1"] -- "path A cost 5" --> R2b["R2"]
    R1b -- "path B cost 5" --> R2b
    R2b -. "FIB has both paths" .-> R1b
    R2b -. "ECMP: 50/50 hashing" .-> R1b
  end

The default maximum-paths 1 is fine for most deployments. The discipline is to raise it when there is parallel capacity that should be used.

How the result is validated

show ip ospf interface                          # per-interface cost (auto or manual)
show ip ospf auto-cost reference-bandwidth      # the global reference-bandwidth
show ip route 10.99.0.0/16                      # specific prefix the operator expects
show ip route ospf                              # all OSPF routes with their cost
traceroute 10.99.0.1                            # validate the path the traffic takes

A working baseline shows:

• Every interface has the expected cost (auto or manual).
• show ip route shows the expected paths for every prefix.
• traceroute follows the expected path through the domain.
• ECMP paths appear in show ip route as multiple via ... lines for the same prefix.

If the operator changed a cost but the path did not change, the most likely cause is: the changed cost is not actually on the path the traffic takes (a transitive link); or the reference-bandwidth on the local router disagrees with the rest of the domain.

How it fails

The production failure modes the engineer must recognise:

• Suboptimal path selection. A 100 Mbps link is preferred over a 1 Gbps link because both have auto-cost 1 (the reference-bandwidth is 100 Mbps). The fix is to raise the reference-bandwidth to 10 Gbps and validate.
• Asymmetric reference-bandwidth. R1 has reference 100 Mbps, R2 has reference 10 Gbps. R1 picks a 100 Mbps path (cost 1); R2 picks a 10 Gbps path (cost 1). Forwarding is asymmetric; the return path may not match the forward path.
• Manual cost on the wrong interface. The operator sets interface eth0 cost 50 intending to deprioritise the primary path; the actual primary path is eth1. No effect; routing still prefers the high-bandwidth link.
• ECMP not engaging. Two paths have equal cost, but the FIB shows only one. maximum-paths is 1 (default). The fix is to raise maximum-paths.
• Route-map metric-set not engaged. The redistribution block has set metric 50 in the route-map, but the LSDB shows the default metric (20). The route-map may not be applied; check the set protocols ospf redistribute bgp route-map PARTNER-INTO-OSPF line.
• Cost on a Type-5 LSA without summarisation. The operator sets a metric on a specific partner prefix, but the prefix is inside an area range (covered in lesson xix-03). The area range produces a single Type-3 LSA at the ABR; the Type-5 from the redistribution is hidden behind the Type-3.

Rollback

The recovery from a bad cost configuration:

• Reference-bandwidth change. set protocols ospf auto-cost reference-bandwidth 100 reverts to the default. The cost recalculates on every interface; SPF reruns; the FIB updates.
• Manual cost removal. delete protocols ospf interface eth0 cost reverts the interface to auto-cost.
• Route-map metric-set removal. Edit the route-map; remove the set metric line. The next Type-5 LSA uses the default metric.
• Maximum-paths reduction. set protocols ospf maximum-paths 1 reduces ECMP to a single path. The FIB removes the redundant paths; one path remains.

For any of these, commit; save applies, and rollback N inside configure reverts to a known-good revision.

Cross-course references

The VyOS lessons vyos-xviii-06-spf-and-cost covers the SPF algorithm and the cost model this lesson builds on. The lesson vyos-xix-02-ospf-interface-config covers the per-interface configuration block where manual cost lives. The lesson vyos-xix-04-ospf-redistribute covers the redistribution block where the route-map metric-set lives. The VyOS lesson on ECMP (Part XXXVI) covers the hashing algorithms that ECMP uses to distribute flows across multiple paths. The Linux course’s XV-Linux-NetConfig covers the kernel routing-table primitives that the cost calculation drives.

Quiz