8.4 ECMP, Static Routing, and VRF Isolation

ECMP, Static Routing, and VRF Isolation

Quick Summary: Modern enterprise networks demand both high-capacity transit performance and strict security segmentation. Equal-Cost Multi-Path (ECMP) balances traffic across redundant uplinks simultaneously using 5-tuple hashing, maximizing throughput while preventing packet reordering. For predictable perimeter connectivity, static routing and floating static routes deliver dependable primary and fallback paths. At the architectural level, Virtual Routing and Forwarding (VRF) virtualizes the switch's routing table into isolated logical routing instances, preventing traffic leakage between corporate, guest, IoT, and management domains at Layer 3.

---

Equal-Cost Multi-Path (ECMP) Routing

In traditional spanning tree environments, redundant Layer 2 links are blocked to prevent bridging loops, leaving expensive bandwidth completely idle. At Layer 3, however, redundant routed paths can be utilized concurrently through Equal-Cost Multi-Path (ECMP).

ECMP Requirements and Operational Mechanics

When a routing protocol (such as OSPF) or static configuration discovers multiple paths to the exact same destination prefix, ECMP activates under three strict conditions:

1. The destination prefix and subnet mask must be identical.
2. The Administrative Distance must be identical (routes learned from the same source).
3. The metric/path cost must be identical.

When these conditions are met, AOS-CX installs the equal-cost next hops into the hardware Forwarding Information Base (FIB), up to the platform's ECMP limit (a static route can have up to 32 next hops).

                               +-------------------------+
                               |   Agg-Switch A (NextHop)|  Cost: 20
                       +-----> |      10.0.1.2/30        | ----------+
                       |       +-------------------------+           |
[ Access Switch ] -----+                                             +-----> [ Core / WAN ]
                       |       +-------------------------+           |
                       +-----> |   Agg-Switch B (NextHop)| ----------+
                               |      10.0.2.2/30        |  Cost: 20
                               +-------------------------+

The 5-Tuple Hashing Algorithm

A critical challenge in multi-path forwarding is ensuring that packets belonging to the same TCP or UDP dialogue do not arrive out of order. If a switch load-balanced packets using simple round-robin distribution (Packet 1 to Path A, Packet 2 to Path B), slight latency differences between links would cause packets to arrive out of sequence, triggering TCP window collapse, retransmissions, and severe application degradation.

To prevent this, AOS-CX switches implement deterministic per-flow 5-tuple hashing in the switch ASIC. The hash function inputs five header fields:

1. Source IP Address
2. Destination IP Address
3. IP Protocol (e.g., TCP = 6, UDP = 17)
4. Source Layer 4 Port
5. Destination Layer 4 Port

Because all packets within a specific TCP/UDP flow have identical 5-tuple values, the mathematical hash generates the exact same numerical result. Consequently, all packets for that flow are consistently forwarded over the same physical next-hop link, guaranteeing sequential packet delivery while evenly distributing hundreds of concurrent client flows across all available paths.

---

Static Routing and Floating Static Routes

While dynamic routing protocols like OSPF are preferred across enterprise core and aggregation layers, static routing remains vital at network perimeters, branch offices, and firewall handoffs.

Standard Static and Default Routes

A static route maps a destination network to a specific next-hop IP address or egress interface:

• Specific Static Route: Directs traffic destined for a particular remote subnet (ip route 10.50.0.0/16 10.0.1.2).
• Default Route (Gateway of Last Resort): Configured with a prefix of 0.0.0.0/0 (ip route 0.0.0.0/0 10.0.1.1). Any packet whose destination address does not match a more specific route in the routing table is forwarded to this default gateway.

Floating Static Routes for Automated Failover

A floating static route is an administratively configured static route whose Administrative Distance is deliberately set higher than that of the primary dynamic routing protocol.

• Under normal conditions, the primary dynamic protocol (e.g., OSPF with AD 110) installs its route into the active routing table.
• Because the floating static route has a higher AD (e.g., AD 120 or 200), it remains dormant in configuration and is not installed in the active RIB.
• If the primary link fails and the OSPF neighbor relationship drops, the OSPF route is removed from the routing table.
• The switch immediately installs the floating static route into the active FIB, restoring connectivity across a backup circuit without manual administrator intervention.

! Configure Primary OSPF on Interface
interface 1/1/48
    routing
    ip address 10.0.1.1/30
    ip ospf 1 area 0

! Configure Floating Static Route with AD 120 (higher than OSPF 110)
ip route 0.0.0.0/0 192.168.100.1 distance 120

---

Virtual Routing and Forwarding (VRF) Architecture

In traditional networking, a switch maintains a single global routing table. If an enterprise needs to isolate Guest Wi-Fi traffic, IoT building automation, and corporate finance systems, traditional solutions relied on complex Access Control Lists (ACLs) applied across shared VLANs. However, shared routing tables still allow attackers to probe switch control plane interfaces and route tables.

Virtual Routing and Forwarding (VRF) provides true Layer 3 multi-tenancy. VRF virtualizes the physical switch into multiple independent logical routers:

• Independent Route Tables (RIB/FIB): Each VRF maintains its own isolated routing table. Routes existing in VRF_GUEST are completely invisible to VRF_CORP.
• Independent ARP Tables: MAC-to-IP bindings are segregated per VRF.
• Overlapping IP Address Spaces: Two different VRFs can use the exact same IP subnets (e.g., both using 192.168.1.0/24) without IP collision, because their forwarding tables are physically separated in hardware ASICs.

+-----------------------------------------------------------------------------------+
|                         AOS-CX VRF SEGREGATION MODEL                              |
|                                                                                   |
|  +--------------------+   +--------------------+   +--------------------+         |
|  |     default VRF    |   |      mgmt VRF      |   |    TENANT_A VRF    |         |
|  | Corporate Data     |   | Out-of-Band Port   |   | Isolated Guest/IoT |         |
|  | Routed SVIs & LAGs |   | Dedicated mgmt port|   | SVI Vlan 100       |         |
|  +--------------------+   +--------------------+   +--------------------+         |
|            |                        |                        |                    |
|            v                        v                        v                    |
|  [ Data Plane ASICs ]     [ Control Plane CPU ]    [ Data Plane ASICs ]           |
+-----------------------------------------------------------------------------------+

Built-in VRFs in Aruba AOS-CX

AOS-CX switches incorporate two pre-defined, non-deletable VRFs out of the box:

1. default VRF: The primary operational VRF for all front-panel data ports, SVIs, and user traffic. Unless assigned elsewhere, all Layer 3 interfaces belong to the default VRF.
2. mgmt VRF: Dedicated strictly to the physical out-of-band management interface (interface mgmt). Isolating management traffic inside the mgmt VRF prevents denial-of-service (DoS) attacks on the data plane from impacting switch management access.

Working with the Management VRF

Because the mgmt VRF is completely isolated from the data plane, standard commands executed in the CLI operate in the default VRF by default. To interact with the management network, administrators must append the vrf mgmt parameter:

! Configure out-of-band management IP address
switch(config)# interface mgmt
switch(config-if-mgmt)# no shutdown
switch(config-if-mgmt)# ip static 192.168.1.50/24
switch(config-if-mgmt)# exit

! Configure the management default gateway (interface mgmt context)
switch(config)# interface mgmt
switch(config-if-mgmt)# default-gateway 192.168.1.1
switch(config-if-mgmt)# exit

! Test reachability via management network
switch# ping 192.168.1.1 vrf mgmt
switch# traceroute 8.8.8.8 vrf mgmt

Creating and Binding Custom User VRFs

Administrators can create custom VRFs to segment specific user communities or business units:

! Step 1: Create custom VRF
switch(config)# vrf GUEST_USERS
switch(config-vrf)# exit

! Step 2: Attach SVI to the VRF
switch(config)# interface vlan 100
switch(config-if-vlan)# description Guest-Gateway
switch(config-if-vlan)# vrf attach GUEST_USERS
switch(config-if-vlan)# ip address 10.100.1.1/24
switch(config-if-vlan)# exit

! Step 3: Configure isolated static default route inside GUEST_USERS VRF
switch(config)# ip route 0.0.0.0/0 10.100.1.254 vrf GUEST_USERS

Configuration Warning: Changing an interface's VRF membership removes its Layer 3 configuration (for example, no vrf attach warns that all Layer 3 configuration associated with the VRF will be deleted). Attach the VRF before configuring IP addresses on the interface.

---

WAN Edge Routing and Campus Fabrics Overview

While static routing and OSPF dominate internal campus aggregation and access layers, modern enterprise campus networks interface with broader wide-area and data center architectures:

• Border Gateway Protocol (BGP): At the campus WAN edge, enterprise border routers peer with Internet Service Providers (ISPs) using External BGP (eBGP, AD 20). Within large multi-site campuses, Internal BGP (iBGP, AD 200) propagates external prefixes across redundant core switches.
• EVPN-VXLAN Campus Fabrics: In advanced campus architectures, Aruba Central and AOS-CX switches implement Ethernet VPN (EVPN) with VXLAN encapsulation (RFC 7348). In this architecture, an underlying physical IP network (the underlay) runs OSPF and ECMP for rapid, resilient packet transport. Over this underlay, switches establish a virtualized overlay network using Multiprotocol BGP (MP-BGP EVPN) as the control plane. Customer VRFs map directly to Layer 3 Virtual Network Identifiers (VNIs), enabling campus-wide macro-segmentation and seamless Layer 2 extension across routed boundaries without Spanning Tree.

---

AOS-CX CLI Verification Commands

The following commands verify ECMP forwarding, static routes, and VRF segregation:

switch# show ip route
Displaying ipv4 routes selected for forwarding
0.0.0.0/0, vrf default
        via 10.0.1.2, [110/20], ospf
        via 10.0.2.2, [110/20], ospf

switch# show ip route vrf GUEST_USERS
Displaying ipv4 routes selected for forwarding
0.0.0.0/0, vrf GUEST_USERS
        via 10.100.1.254, [1/0], static
10.100.1.0/24, vrf GUEST_USERS
        via vlan100, [0/0], connected

Key Verification Table

---

Common Exam Traps

• VRF Order of Operations: Changing an interface's VRF membership deletes its Layer 3 configuration, so attach the VRF first and then configure the IP address.
• Management Default Gateway: ip route 0.0.0.0/0 192.168.1.1 installs a route in the default VRF and does nothing for the management port. Give the management interface its gateway with default-gateway 192.168.1.1 under interface mgmt, and test it with ping 192.168.1.1 vrf mgmt.
• Floating Static Route Distance: To serve as a backup to OSPF (AD 110), the static route's administrative distance must be set greater than 110 (such as 120 or 200). Setting the static distance to 10 or leaving it at the default of 1 will override OSPF completely rather than acting as a backup.