8.1 IP Routing Foundations & Administrative Distance

8.1 IP Routing Foundations & Administrative Distance

Modern enterprise networks rely on high-speed Layer 3 routing to interconnect campus distribution blocks, data center fabrics, SD-WAN edge nodes, and cloud transit gateways. To engineer resilient, deterministic routing architectures and troubleshoot complex path selection anomalies, enterprise network engineers must master the internal mechanics of packet forwarding: how routers store and compile routing information, how the forwarding plane evaluates candidate paths, and how administrative controls influence traffic steering.

+-----------------------------------------------------------------------------------+
|                         LAYER 3 PACKET FORWARDING ENGINE                          |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  CONTROL PLANE (Software / CPU)                                                   |
|  +------------+  +------------+  +------------+  +------------+  +-------------+  |
|  | Connected  |  |   Static   |  |    OSPF    |  |   EIGRP    |  |     BGP     |  |
|  |   (AD 0)   |  |   (AD 1)   |  |  (AD 110)  |  |  (AD 90)   |  | (AD 20/200) |  |
|  +------------+  +------------+  +------------+  +------------+  +-------------+  |
|        \               |               |               |                /         |
|         +--------------+---------------+---------------+---------------+          |
|                                        |                                          |
|                                        v                                          |
|                    [ Routing Information Base (RIB) ]                             |
|                    - Evaluates AD for identical prefixes                          |
|                    - Resolves recursive next-hops                                 |
|                                        |                                          |
+----------------------------------------|------------------------------------------+
|                                 CEF Download                                      |
|                                        v                                          |
|  DATA PLANE (Hardware / ASICs / TCAM)                                             |
|  +-----------------------------------+   +-------------------------------------+  |
|  |  Forwarding Information Base      |   |           Adjacency Table           |  |
|  |               (FIB)               |   |  - Pre-computed Layer 2 rewrite     |  |
|  |  - Flat, non-recursive prefix map |<--|    headers (Dest MAC, Src MAC, Type)|  |
|  |  - Optimized 4-level mtrie tree   |   |  - Interface output pointers        |  |
|  +-----------------------------------+   +-------------------------------------+  |
|                                        |                                          |
|                                        v                                          |
|  [ Line-Rate Packet Forwarding via ASIC Ternary Content-Addressable Memory ]      |
+-----------------------------------------------------------------------------------+

1. Control Plane vs. Data Plane: RIB, FIB, and CEF Architecture

Cisco IOS-XE separates router operations into two primary architectural domains: the Control Plane and the Data (Forwarding) Plane. Understanding the division of responsibility between these planes is vital for diagnosing routing convergence and packet-forwarding discrepancies.

The Control Plane & Routing Information Base (RIB)

The control plane is executed by the central Route Processor (CPU). It runs dynamic routing protocol processes (OSPF, EIGRP, BGP, IS-IS), processes routing updates, handles neighbor adjacencies, and maintains the protocol-specific topology databases (e.g., OSPF Link-State Database, EIGRP Topology Table, BGP Table).

• RIB Function: The Routing Information Base (RIB), commonly referred to as the IP routing table (show ip route), is a centralized control-plane database managed by the Route Processor. It aggregates the best route candidates from every active routing protocol, static configuration, and directly connected interface.
• Recursive Routing Resolution: The RIB often contains recursive entries where a destination prefix points to a next-hop IP address rather than a physical egress interface (e.g., a BGP route pointing to a loopback address learned via OSPF). The control plane must perform recursive lookups to resolve the ultimate outgoing interface.
• Control Plane Limitation: Looking up destinations in the RIB requires software CPU cycles and hash table walks. If packets were routed directly by the control plane, router throughput would drop by orders of magnitude (process switching).

The Data Plane & Cisco Express Forwarding (CEF)

To achieve line-rate packet switching across multi-gigabit and terabit interfaces, Cisco IOS-XE uses Cisco Express Forwarding (CEF). CEF compiles the dynamic information generated in the control plane into two highly optimized hardware data structures instantiated in Application-Specific Integrated Circuit (ASIC) memory: the Forwarding Information Base (FIB) and the Adjacency Table.

+-----------------------------------------------------------------------------------+
|                        CISCO EXPRESS FORWARDING (CEF) LOOKUP                      |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  Incoming Packet: [ Dest IP: 10.1.1.50 ]                                          |
|                          |                                                        |
|                          v                                                        |
|  +-------------------------------------------------------------+                  |
|  | Forwarding Information Base (FIB) - Hardware TCAM Lookup    |                  |
|  | Prefix Match: 10.1.1.0/24 -> Adjacency Pointer: Index #0x4A |                  |
|  +-------------------------------------------------------------+                  |
|                          |                                                        |
|                          v                                                        |
|  +-------------------------------------------------------------+                  |
|  | Adjacency Table - Layer 2 Frame Rewrite Construction        |                  |
|  | Index #0x4A: Outgoing Interface: GigabitEthernet0/0/1       |                  |
|  |              Dest MAC: 00:50:56:AB:CD:EF (Next-Hop Host)    |                  |
|  |              Src MAC:  00:1E:BD:12:34:56 (Local Egress MAC) |                  |
|  |              EtherType: 0x0800 (IPv4)                       |                  |
|  +-------------------------------------------------------------+                  |
|                          |                                                        |
|                          v                                                        |
|  Outgoing Frame: [ L2 Header Rewrite ] + [ IP Packet Payload ]                    |
+-----------------------------------------------------------------------------------+

1. Forwarding Information Base (FIB)

• The FIB is a flat, non-recursive mirror of the RIB, optimized for lightning-fast hardware searches using a 256-way multi-branch tree (mtrie) structure stored in Ternary Content-Addressable Memory (TCAM).
• All recursive next-hops from the RIB are completely resolved prior to FIB programming. When an incoming packet is processed, the hardware ASIC performs a single TCAM lookup to match the destination IP against the FIB and immediately obtain the index pointer to the Adjacency Table.

2. Adjacency Table

• The Adjacency Table maintains pre-computed Layer 2 next-hop framing and encapsulation headers for all connected neighbors and next-hops, derived directly from the ARP table (IPv4) or Neighbor Discovery Protocol (NDP for IPv6).
• Decoupling Rationale: Decoupling the Layer 2 rewrite information (Adjacency Table) from the IP prefix database (FIB) ensures that when a next-hop's MAC address changes or an ARP entry refreshes, the router updates only a single adjacency pointer without needing to reprogram hundreds of thousands of prefix entries in the TCAM FIB.

Evolution of Cisco Switching Mechanisms

! Verify the Control-Plane Routing Information Base (RIB)
Router# show ip route 10.1.1.50
Routing entry for 10.1.1.0/24
  Known via "ospf 1", distance 110, metric 20, type intra area
  Last update from 192.168.12.2 on GigabitEthernet0/0/1, 00:14:22 ago
  Routing Descriptor Blocks:
  * 192.168.12.2, from 192.168.12.2, 00:14:22 ago, via GigabitEthernet0/0/1
      Route metric is 20, share count 1

! Verify the Hardware Forwarding Information Base (FIB) entry
Router# show ip cef 10.1.1.50
10.1.1.0/24
  nexthop 192.168.12.2 GigabitEthernet0/0/1

! Verify the Layer 2 Adjacency Table entry
Router# show adjacency GigabitEthernet0/0/1 detail
Protocol Interface                 Address
IP       GigabitEthernet0/0/1      192.168.12.2(7)
                                   005056ABCDEF001EBD1234560800
                                   epoch 0
                                   sourced: IP adj out of GigabitEthernet0/0/1
                                   Encap length 14
                                   005056ABCDEF001EBD1234560800
                                   L3 mtu 1500
                                   Next hop 192.168.12.2

2. The Path Selection Hierarchy & Longest Prefix Match (LPM)

When a router forwards an IP packet, it evaluates candidate paths through a strict three-tier decision hierarchy. Confusing these three distinct selection steps is the single most common conceptual trap on the ENCOR exam.

+-----------------------------------------------------------------------------------+
|                         THE 3-TIER PATH SELECTION HIERARCHY                       |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  STEP 1: LONGEST PREFIX MATCH (LPM) [Forwarding Plane / FIB]                      |
|  - Which route has the longest prefix length (most specific / highest mask bits)? |
|  - Evaluated across ALL routing sources simultaneously.                           |
|  - WINNER FORWARDS THE PACKET. If match is unique, stop here!                     |
|                                                                                   |
|  STEP 2: ADMINISTRATIVE DISTANCE (AD) [Control Plane / RIB]                       |
|  - If multiple sources advertise the EXACT SAME prefix AND mask length:          |
|  - Which source has the lowest Administrative Distance (trustworthiness)?         |
|  - WINNER IS INSTALLED IN THE RIB / FIB.                                          |
|                                                                                   |
|  STEP 3: PROTOCOL METRIC [Inside Individual Routing Protocol]                     |
|  - If the SAME routing protocol learns multiple paths to the SAME prefix/mask:   |
|  - Which path has the lowest protocol-specific metric (cost, composite, etc.)?   |
|  - WINNER IS OFFERED TO THE RIB (or multiple paths if ECMP/UCMP).                 |
+-----------------------------------------------------------------------------------+

Longest Prefix Match (LPM) Mechanics

Longest Prefix Match (LPM) is the fundamental law of IP packet forwarding. When an IP packet arrives at an interface, the forwarding engine scans the FIB for all prefixes that encompass the destination IP address. The router always selects the route with the longest prefix length (the largest prefix mask number / most specific network bit match).

CRITICAL RULE: Longest Prefix Match operates independently of Administrative Distance and Metric. A route learned via an untrusted or high-AD protocol with a more specific mask (e.g., OSPF /28, AD 110) will always take precedence over a less specific route learned via a low-AD protocol (e.g., Static /24, AD 1, or Connected /16, AD 0).

LPM Evaluation Walkthrough

Consider a router whose routing table contains the following five distinct entries:

Entry 1: 0.0.0.0/0          via 203.0.113.1   [Static Default, AD 1, Metric 0]
Entry 2: 172.16.0.0/16      via 192.168.10.1  [BGP, AD 20, Metric 0]
Entry 3: 172.16.10.0/24     via Gigabit0/0/0  [Directly Connected, AD 0, Metric 0]
Entry 4: 172.16.10.64/28    via 192.168.20.1  [OSPF, AD 110, Metric 45]
Entry 5: 172.16.10.65/32    via 192.168.30.1  [EIGRP, AD 90, Metric 130560]

Now observe how the router evaluates four distinct destination IP addresses:

3. Administrative Distance (AD) Architecture

Administrative Distance (AD) is an integer value from 0 to 255 that defines the administrative believability (trustworthiness) of a routing information source.

When is Administrative Distance Evaluated?

Administrative Distance is invoked only in the control plane when the Routing Information Base (RIB) receives two or more candidate routes from different routing sources for the exact same destination prefix and prefix length (e.g., OSPF advertises 10.5.0.0/16 and EIGRP also advertises 10.5.0.0/16).

• The routing source with the lowest AD integer wins and is installed into the RIB (and subsequently downloaded into the CEF FIB).
• The losing route is maintained in the losing protocol's private database (e.g., OSPF LSDB) as a dormant backup, ready to be promoted if the winning route withdraws.

+-----------------------------------------------------------------------------------+
|                       ADMINISTRATIVE DISTANCE COMPARISON                          |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  Routing Source A: EIGRP Internal  ---> [ 10.5.0.0/16 ] (AD: 90)   --- WINNER!    |
|                                                                    (Installed)    |
|  Routing Source B: OSPF            ---> [ 10.5.0.0/16 ] (AD: 110)  --- Loser      |
|                                                                    (In LSDB)      |
|  Routing Source C: RIPv2           ---> [ 10.5.0.0/16 ] (AD: 120)  --- Loser      |
|                                                                                   |
+-----------------------------------------------------------------------------------+

Comprehensive Cisco Administrative Distance Hierarchy

Modifying Administrative Distance

Administrators can override default AD values on a per-protocol, per-prefix, or per-neighbor basis using the distance configuration command. This technique is commonly employed during routing migrations and mutual route redistribution to prevent suboptimal routing loops.

! Modify administrative distance for OSPF routes
Router(config)# router ospf 1
Router(config-router)# distance 105                   ! Set all OSPF routes to AD 105
Router(config-router)# distance ospf intra-area 95 inter-area 100 external 175

! Modify administrative distance for EIGRP routes
Router(config)# router eigrp 100
Router(config-router)# distance eigrp 85 165          ! Internal AD 85, External AD 165

Local Scope Principle: Administrative Distance is strictly local to the router on which it is configured. It is never included, signaled, or carried inside routing update packets (OSPF LSAs, EIGRP updates, or BGP UPDATE messages) to neighboring routers.

4. Floating Static Routes & Resilient Failover Design

A Floating Static Route is a static route configured with an Administrative Distance intentionally set higher than the AD of the active dynamic routing protocol. It remains dormant and invisible in the active routing table as long as the primary dynamic route is present. If the primary route drops out of the RIB due to a link failure or neighbor outage, the floating static route immediately "floats" into the RIB to provide seamless backup connectivity.

+-----------------------------------------------------------------------------------+
|                         FLOATING STATIC ROUTE FAILOVER                            |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|                     [ Primary Path: OSPF (AD 110) - ACTIVE ]                      |
|                    /                                        \                     |
|  [ Local Branch ] =                                          = [ Enterprise Core ]|
|                    \                                        /                     |
|                     [ Backup Path: Static (AD 115) - DORMANT ]                    |
|                                                                                   |
|  1. NORMAL STATE: OSPF route (AD 110) installed in RIB. Static (AD 115) inactive. |
|  2. PRIMARY LINK FAILS: OSPF neighbor down -> OSPF route withdrawn from RIB.      |
|  3. FAILOVER: Static route (AD 115) immediately installed in RIB and CEF FIB.     |
+-----------------------------------------------------------------------------------+

The Silent Link Failure Trap & IP SLA Tracking

In modern switched topologies (e.g., Ethernet WAN circuits or carrier demarcation points), a physical link to a local Layer 2 switch may remain in the up/up state even when the service provider's upstream transit network has completely failed.

Because standard static routes remain active in the RIB as long as the outgoing interface is physically up, a standard floating static route pointing to an unreachable next-hop will continue forwarding traffic into a "black hole." To resolve this vulnerability, engineers pair floating static routes with IP Service Level Agreement (IP SLA) probing and Object Tracking.

! Step 1: Configure an IP SLA ICMP Echo Probe to test end-to-end reachability
ip sla 10
 icmp-echo 203.0.113.1 source-interface GigabitEthernet0/0/2
 threshold 1000       ! Alert if latency exceeds 1000ms
 timeout 2000         ! Timeout after 2000ms
 frequency 5          ! Probe every 5 seconds
exit

! Step 2: Schedule the IP SLA probe to run continuously
ip sla schedule 10 life forever start-time now

! Step 3: Configure an Object Track to monitor the operational state of IP SLA 10
track 1 ip sla 10 reachability
 delay down 10 up 15  ! Hysteresis: wait 10s before failing down, 15s before restoring
exit

! Step 4: Associate the primary static route with Track 1 and define the floating backup
! Primary Static Route (AD 1, Active only when Track 1 is UP)
ip route 0.0.0.0 0.0.0.0 203.0.113.1 track 1

! Floating Static Backup Route via LTE Gateway (AD 210, Installed if Primary fails)
ip route 0.0.0.0 0.0.0.0 198.51.100.1 210

5. Multi-Path Forwarding: ECMP vs. UCMP

When a router possesses multiple valid paths to a single destination prefix, it can distribute packet flows across multiple interfaces simultaneously to optimize aggregate bandwidth utilization.

Equal-Cost Multi-Pathing (ECMP)

Equal-Cost Multi-Pathing (ECMP) occurs when a single routing protocol discovers multiple paths to the exact same prefix with identical Administrative Distance and identical metric.

• Platform Support: Supported natively by all major routing protocols (OSPF, EIGRP, BGP, IS-IS) and static routing.
• Maximum Paths: Cisco IOS-XE supports up to 4 paths by default (configurable up to 16, 32, or 64 paths depending on platform via the maximum-paths <number> command).
• Hardware Flow Hashing: CEF prevents out-of-order packet delivery by assigning entire TCP/UDP sessions to a specific link using a hardware hash algorithm. The hash is typically computed from the packet's 5-tuple:
  1. Source IP Address
  2. Destination IP Address
  3. Protocol Field (TCP / UDP / ICMP)
  4. Source Layer 4 Port
  5. Destination Layer 4 Port

+-----------------------------------------------------------------------------------+
|                        EQUAL-COST MULTI-PATHING (ECMP)                            |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|                           +--- Path 1 (Cost 10) ---> [50% of traffic flows]       |
|  [ Ingress Router ] ===== |                                                       |
|                           +--- Path 2 (Cost 10) ---> [50% of traffic flows]       |
|                                                                                   |
+-----------------------------------------------------------------------------------+

Unequal-Cost Multi-Pathing (UCMP)

Unequal-Cost Multi-Pathing (UCMP) allows a router to load-balance traffic proportionally across multiple paths with different metrics.

• Protocol Exclusivity: In Cisco enterprise networks, EIGRP is the only Interior Gateway Protocol (IGP) that supports true Unequal-Cost Multi-Pathing.
• The Variance Multiplier: EIGRP implements UCMP via the variance <n> command (where $n$ is an integer from 1 to 128, default: 1).
• The Two Mandatory Prerequisites for UCMP:
  1. Feasibility Condition Rule: The candidate path must be a valid Feasible Successor. Its Reported Distance (RD) must be strictly less than the Feasible Distance (FD) of the Successor ($RD < FD_{successor}$). Non-feasible paths can never participate in UCMP, regardless of variance.
  2. Variance Threshold Rule: The total metric of the Feasible Successor path must be less than or equal to the Feasible Distance of the Successor multiplied by the variance integer: $\text{Metric}_{\text{Candidate}} \le \text{FD}_{\text{Successor}} \times \text{Variance}$

+-----------------------------------------------------------------------------------+
|                       UNEQUAL-COST MULTI-PATHING (UCMP)                           |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|                           +--- Path 1 (Successor: Metric 20)  ---> [60% Flows]    |
|  [ Ingress Router ] ===== |                                                       |
|   (Variance = 2)          +--- Path 2 (Feasible Successor:    ---> [40% Flows]    |
|                                        Metric 30 <= 20 * 2)                       |
|                                                                                   |
+-----------------------------------------------------------------------------------+

UCMP Traffic Distribution Calculation

Traffic is load-balanced across active UCMP paths in inverse proportion to their total metrics. Paths with lower metrics receive proportionally more traffic flows:

$\text{Traffic Ratio}_A : \text{Traffic Ratio}_B = \frac{1}{\text{Metric}_A} : \frac{1}{\text{Metric}_B}$

For example, if Path 1 has a metric of 20,000 and Path 2 (a valid Feasible Successor) has a metric of 40,000 with variance 2 configured, traffic will be distributed in a 2:1 ratio (66.6% across Path 1, 33.3% across Path 2).

Architectural Summary: ECMP vs. UCMP