IGP versus EGP — the inside and outside of an Autonomous System

Routing protocols divide into two categories that map to an organisational boundary in the network. Interior Gateway Protocols (IGPs) — OSPF, IS-IS, RIP — run inside an organisation. Exterior Gateway Protocols (EGPs) — BGP, the only one in common use — run between organisations. The boundary between the two is the Autonomous System, an administrative and policy concept that defines whose routing decisions the router is executing.

This lesson is about the boundary — what an Autonomous System is, why the boundary exists, what protocols live on each side, and the production failure modes that come from putting the boundary in the wrong place.

What an Autonomous System is

An Autonomous System (AS) is a group of IP networks and routers under the control of a single organisation that presents a single routing policy to the rest of the Internet. The AS is identified by a number — an ASN — assigned by IANA and delegated through the regional registries.

ASNs were originally 16 bits wide, so the space ran from 0 to 65535, with 64512–65534 reserved for private use. RFC 6793 widened the field to 32 bits, so the space now runs 0 to 4294967295; RFC 6996 reserves 4200000000–4294967294 as the 32-bit private range, and 23456 is permanently reserved as AS_TRANS, the placeholder a 16-bit-only speaker substitutes when it cannot represent a 32-bit ASN in the AS_PATH.

VyOS accepts the full 32-bit range on set protocols bgp system-as. Every example in this course uses a documentation or private ASN (64512 and above) so that nothing here can be pasted onto the public Internet by accident.

flowchart TD
  subgraph AS65001
    R1[Router R1] --- R2[Router R2]
    R2 --- R3[Router R3]
    R3 --- R4[Router R4]
    R1 --- R4
  end
  AS65001 -->|eBGP| AS65002
  AS65001 -->|eBGP| AS65003

The diagram shows the canonical shape: an AS has multiple internal routers connected by an IGP; the AS connects to neighbouring ASes with eBGP. The boundary routers — sometimes called edge routers or peering routers — run both IGP (toward the inside) and eBGP (toward the outside).

The AS is not a physical boundary — there is no wire that separates inside from outside. The AS is an administrative boundary: the set of prefixes the AS advertises, the policies it applies to incoming and outgoing routes, and the routing decisions it makes on behalf of its constituents. Different organisations have different priorities (some optimise for cost, some for diversity, some for latency), and the AS is the right granularity at which to express those priorities.

The IGP / EGP split — why it exists

The split exists because the routing problems inside an AS are different from the routing problems between ASes:

• Inside an AS, the operator controls every router. The protocols can trust the routers; they can flood LSAs freely; they can assume every router is reachable from every other. The dominant cost is convergence: how fast the network reconverges after a topology change.
• Between ASes, the operator does not control the other side. The protocol must be policy-rich: it must allow one AS to express preference for inbound traffic, another to express preference for outbound traffic, a third to refuse to transit. The dominant cost is policy: who is allowed to reach whom, and through which path.

IGPs are designed for the first problem. They are link-state or distance-vector protocols that converge quickly and optimise for shortest path. They have no rich policy mechanism; they assume the operator wants to reach every prefix through the cheapest path inside the AS.

BGP is designed for the second problem. It is a slow, policy-rich path-vector protocol that optimises for policy compliance rather than shortest path. Its attributes — LOCAL_PREF, MED, COMMUNITY, AS_PATH, ORIGIN, NEXT_HOP — exist almost entirely to express policy, and its best-path algorithm consults them long before it looks at anything resembling a distance.

IGP protocols: OSPF and IS-IS

Two IGPs dominate modern production: OSPF (RFC 2328) and IS-IS (ISO 10589, with the IP extensions of RFC 1195). Both are link-state protocols that build a topology database per area or level, run SPF against the database, and install the best paths. The differences are operational:

• OSPF is carried in IP, as IP protocol number 89. On VyOS it is configured under set protocols ospf, per area and per interface. Areas are designed around a backbone (area 0) with summarisation at the Area Border Routers.
• IS-IS is not carried in IP at all. Its PDUs ride directly on the data link as OSI (CLNS) traffic, which is why an IS-IS router is addressed by a NET rather than by an IP address, and why IS-IS keeps working on a link whose IP addressing is broken. It is the IGP of choice in large service-provider networks: its two-level hierarchy is easier to grow than OSPF’s single-backbone constraint, and its TLV-based PDUs extend without a protocol revision.

flowchart LR
  R1[Router R1] -->|OSPF area 0| R2[Router R2]
  R2 -->|OSPF area 0| R3[Router R3 ABR]
  R3 -->|OSPF area 1| R4[Router R4]
  R4 -->|OSPF area 1| R5[Router R5]

OSPF areas are hierarchical. Area 0 (the backbone) connects all other areas; inter-area routing happens via Area Border Routers (ABRs); summarisation happens at ABRs to keep area databases manageable. The discipline is to design the area plan before deploying OSPF — areas are not fluid, and the backbone constraint means the network topology must support the area plan.

The minimal OSPF configuration on VyOS 1.5 attaches networks to an area and lets the interfaces form adjacencies:

[edit]
vyos@vyos# set protocols ospf parameters router-id 10.255.0.1
[edit]
vyos@vyos# set protocols ospf area 0 network 10.0.0.0/30
[edit]
vyos@vyos# set protocols ospf area 0 network 10.0.1.0/30
[edit]
vyos@vyos# set protocols ospf interface eth0 authentication md5 key-id 1 md5-key SECRET
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# save

RIP is also an IGP but is rare in production: its 15-hop diameter limit and its slow, timer-driven convergence confine it to small networks. EIGRP is a fourth option in the wider industry, and FRR does ship an EIGRP daemon — but VyOS documents no EIGRP configuration tree, so it is not a protocol you would plan a VyOS network around.

EGP protocols: BGP and only BGP

BGP is the only EGP in production use on the public Internet today. The protocol it replaced was literally called EGP; BGP took over in the early 1990s. BGP is a path-vector protocol: rather than carrying a cost, each BGP UPDATE carries the AS_PATH — the sequence of ASNs the route has traversed. The receiving router uses AS_PATH both to detect routing loops and to apply policy.

BGP runs over TCP port 179, and leans on TCP for ordering and retransmission rather than carrying its own. Liveness is a KEEPALIVE timer inside the session: FRR’s defaults are a 60-second keepalive and a 180-second hold time, and the session is torn down if nothing arrives within the hold time. The negotiated hold time is the lower of the two peers’ values, so your local timer setting is a proposal, not a guarantee.

The canonical eBGP peering configuration on VyOS 1.5:

[edit]
vyos@vyos# set protocols bgp system-as 65001
[edit]
vyos@vyos# set protocols bgp parameters router-id 10.255.0.1
[edit]
vyos@vyos# set protocols bgp neighbor 198.51.100.2 remote-as 65002
[edit]
vyos@vyos# set protocols bgp neighbor 198.51.100.2 description "Upstream A"
[edit]
vyos@vyos# set protocols bgp neighbor 198.51.100.2 address-family ipv4-unicast
[edit]
vyos@vyos# set protocols bgp address-family ipv4-unicast network 203.0.113.0/24
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# save

Read the tree carefully, because its shape is the thing operators get wrong when they arrive from older material. The local ASN lives at system-as — it is a property of the BGP process, not a level in the path. Peers hang off protocols bgp neighbor. What the router originates is configured under protocols bgp address-family ipv4-unicast network. What a specific peer is allowed to send and receive is configured under protocols bgp neighbor <ip> address-family ipv4-unicast, which is where per-peer prefix-lists and route-maps attach.

The boundary in production networks

The IGP / EGP split manifests in three production patterns:

1. Single-homed stub AS. The AS has one upstream. The internal network runs an IGP (often just OSPF, sometimes static). The edge runs eBGP with the upstream. The IGP carries internal prefixes; eBGP carries the default route inward and the AS’s own prefixes outward. The boundary is the edge router.

2. Multi-homed stub AS. The AS has two or more upstreams. The internal network runs an IGP. The edge runs eBGP with each upstream and iBGP between the edge routers, so each edge sees what the other has learned. The boundary is the edge router pair; iBGP is what keeps their views of the outside world from diverging.

3. Transit AS. The AS is itself an upstream for other ASes. The internal network runs an IGP. The edge runs eBGP with customers, peers, and upstreams, plus iBGP to coordinate. The edge is the policy-rich layer; the IGP is the fast-convergence layer underneath.

The boundary is also the trust boundary. IGP messages are trusted implicitly, because every router is in the same organisation. BGP messages from outside the AS are untrusted and must be filtered, bounded, and authenticated. VyOS exposes the session-level defence as set protocols bgp neighbor 198.51.100.2 password SECRET — TCP-MD5 as specified in RFC 2385. That protects the TCP session from injection; it does nothing about what a legitimately-authenticated peer sends you, which is what prefix filters are for.

Design boundaries: where each protocol is right

A short decision table:

The line moves with the topology. A network with two routers can run static routes; a network with twenty routers should run OSPF; a network that interconnects with other organisations must run BGP at the boundary.

How the result is validated

OSPF adjacency first. An adjacency that is not Full is not carrying routes, whatever the interface counters say:

$ show ip ospf neighbor

Neighbor ID     Pri State           Up Time         Dead Time Address         Interface                        RXmtL RqstL DBsmL
10.255.0.2        1 Full/DR         01:12:44          38.412s 10.0.0.2        eth0:10.0.0.1                        0     0     0
10.255.0.3        1 Full/Backup     01:12:39          35.907s 10.0.1.2        eth1:10.0.1.1                        0     0     0

Then the BGP sessions. The column an operator reads first is State/PfxRcd: a number means the session is established and that many prefixes were accepted; a word (Idle, Connect, Active) means it is not:

$ show ip bgp summary

IPv4 Unicast Summary (VRF default):
BGP router identifier 10.255.0.1, local AS number 65001 vrf-id 0
BGP table version 12
RIB entries 7, using 1344 bytes of memory
Peers 2, using 1447 KiB of memory

Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
198.51.100.2    4      65002       842       845        0    0    0 07:01:12            3        1 Upstream A
203.0.113.1     4      65003       839       841        0    0    0 07:00:58            3        1 Upstream B

Total number of neighbors 2

Finally the boundary check itself: for a prefix that both the IGP and BGP know about, which one did zebra actually install?

$ show ip route 10.20.0.0/16

Routing entry for 10.20.0.0/16
Known via "ospf", distance 110, metric 20, best
Last update 00:14:02 ago
* 10.0.0.2, via eth0, weight 1

Routing entry for 10.20.0.0/16
Known via "bgp", distance 20, metric 0
Last update 00:02:11 ago
  198.51.100.2, via eth1, weight 1

Two things to read here. The best keyword and the * mark the route zebra selected and pushed to the kernel; the block without them is a candidate that lost. And the loser in this example is the BGP route despite its lower distance — which means somebody has changed a distance, because on defaults a BGP route at 20 beats an OSPF route at 110 every time. An internal prefix arriving over eBGP and winning is the signature of a route leak somewhere upstream, and is worth an alert rather than a shrug. The mechanics of that selection are the subject of the next lesson.

How it fails

The production failure modes the engineer must recognise:

• Missing iBGP between eBGP edges. Two edge routers, each with its own upstream, and no session between them. Neither edge knows what the other learned, so each makes best-path decisions on half the information, and the AS presents two different pictures of itself to the Internet. Traffic arrives at one edge for a destination only the other edge can reach, and the packets die inside your own AS.
• iBGP full mesh breaking at scale. A hundred iBGP speakers need 4,950 sessions. Long before that becomes unmanageable to configure it becomes unmanageable to change: adding one router touches every other router. The fix is a route reflector pair or a confederation.
• BGP MED oscillation. A multi-homed AS receives the same prefix from two upstreams with different MEDs. Because MED is only comparable within a neighbour AS, the comparison can depend on the order paths happen to sit in the table, and the best path flips on every churn event. The upstream sees flap and may dampen the prefix. The fix is set protocols bgp parameters deterministic-med — which forces the comparison to be order-independent — and, if the design really does want MED compared across neighbour ASes, set protocols bgp parameters always-compare-med on both sides as a deliberate agreement.
• IGP redistributed into BGP without a filter. An OSPF instance has 10,000 internal prefixes and the operator writes set protocols bgp address-family ipv4-unicast redistribute ospf with nothing attached. BGP now advertises all 10,000 to the upstream, blowing through the upstream’s own maximum-prefix limit; the upstream tears the session down and the AS loses transit. The fix is to attach a route-map to the redistribution (... redistribute ospf route-map ONLY-MINE) or, better, to originate a small number of aggregates with network statements and never redistribute an IGP into BGP at all.
• BGP redistributed into OSPF. The inverse, and worse. Injecting a full table into an IGP overwhelms the LSDB, and even a partial injection gives internal routers OSPF paths to external destinations that outrank what the edge learned over eBGP, capturing traffic that should have left the AS. If it has to be done at all, it is done with a route-map that matches a tag, sets a metric deliberately, and denies everything else.

Rollback

A bad IGP / EGP configuration is undone with targeted delete commands, and every one of them is a routing event rather than a cosmetic change:

• Wrong OSPF network statement: delete protocols ospf area 0 network 10.0.1.0/30, then commit. The adjacencies on the interfaces that matched that network tear down.
• Wrong BGP neighbour: delete protocols bgp neighbor 198.51.100.2, then commit. The session tears down and every prefix learned from it is withdrawn.
• Wrong per-peer filter: delete protocols bgp neighbor 198.51.100.2 address-family ipv4-unicast prefix-list export, then commit. The peer now receives everything the router is prepared to advertise, which is exactly the state the filter existed to prevent — so this particular rollback is usually the wrong move, and replacing the filter is the right one.

Production discipline

Additional discipline:

• Run iBGP between every eBGP edge router — the iBGP session is the mechanism by which the AS’s edges agree on what the outside world looks like.
• Filter inbound and outbound on every eBGP session, and bound each session with maximum-prefix. The filter is the AS’s contract with the rest of the Internet; the limit is what survives the filter being wrong.
• Authenticate BGP sessions. set protocols bgp neighbor <ip> password <secret> is TCP-MD5, which is what VyOS exposes and what most transit providers will agree to. It authenticates the session, not the prefixes — RPKI origin validation is the separate control for that.
• Monitor BGP session state on every edge. A session-down event is an alert, not a status line somebody notices on Monday.

Cross-course references

The OSPF course (XVIII-VyOS-OSPFFund through XXII-VyOS-OSPFTroubleshoot) covers IGP in depth. The BGP course (XXIII-VyOS-BGPFund through XXXI-VyOS-BGPTroubleshoot) covers the EGP boundary in depth. The route filtering course (XXVIII-VyOS-BGPPrefixFilters) covers the filtering discipline at the boundary. The route reflector course (XXX-VyOS-BGPRouteReflectors) covers iBGP scaling. The OPNsense course’s XXX-OPNsense-DynamicRouting covers the equivalent FRR-managed IGP / EGP split on the OPNsense platform.

Quiz