3.2 SD-WAN Policies, Traffic Engineering & Application-Aware Routing
Key Takeaways
- The Overlay Management Protocol (OMP) is an MP-BGP-like control protocol running over DTLS/TLS that advertises OMP routes (vRoutes), TLOC routes, and Service routes across the fabric.
- A Transport Location (TLOC) is defined by the key triplet of System IP, Color, and Encapsulation (IPsec/GRE), dictating the underlay attachment points where overlay IPsec tunnels terminate.
- Centralized Control Policies execute on vSmart to control topology (full mesh vs hub-and-spoke) and routing, whereas Centralized Data Policies execute on edge data planes to control traffic steering, Direct Internet Access (DIA), and QoS.
- Application-Aware Routing (AAR) tracks real-time BFD loss, latency, and jitter measurements over IPsec tunnels to dynamically steer application traffic to transport paths satisfying SLA thresholds.
1. Overlay Management Protocol (OMP) Architecture
The Overlay Management Protocol (OMP) is the core control plane protocol of the Cisco Catalyst SD-WAN fabric. Operating similarly to BGP and LISP, OMP runs inside secure DTLS/TLS control plane tunnels between WAN edge routers and centralized vSmart controllers.
OMP eliminates the need for edge routers to peer directly with one another for routing. Instead, edge routers peer only with vSmart controllers ($O(N)$ control connections), and vSmart acts as a centralized route reflector and policy engine.
+-----------------------------------------------------------------------------------+
| OMP ADVERTISED ROUTE TYPES |
+-----------------------------------------------------------------------------------+
1. OMP Routes (vRoutes):
- Prefixes learned from service-side (LAN) networks (Connected, Static, OSPF, BGP).
- Attributes: Prefix, VPN ID, TLOC, Origin Protocol, Origin Metric, Preference, Tag.
|
2. TLOC (Transport Location) Routes:
- Underlay attachment points defining where overlay IPsec data tunnels terminate.
- Key Triplet: [ System IP | Color | Encapsulation (IPsec/GRE) ]
- Attributes: Public/Private IP & Port, TLOC Preference, Weight, Site ID, SPI keys.
|
3. Service Routes:
- Advertises localized network services (Firewall, IPS, NAT, IDP) available at a site.
- Attributes: Service ID (e.g., 'svc-id 1' for FW), VPN ID, TLOC, Site ID.
- Enables service insertion / service chaining without manual IP re-addressing.
+-----------------------------------------------------------------------------------+
The OMP Best-Path Selection Algorithm
When a vSmart controller or WAN edge router receives multiple OMP paths for the same prefix, it evaluates them sequentially using the following deterministic path selection criteria:
+---------------------------------------------------------------------------+
| OMP BEST-PATH SELECTION ALGORITHM |
+---------------------------------------------------------------------------+
| 1. Valid & Reachable TLOC --> Next-hop TLOC must be active and resolved |
| 2. Highest TLOC Preference --> Higher TLOC route preference wins |
| 3. Highest OMP Preference --> Higher OMP route preference wins |
| 4. Lowest Origin Protocol AD--> Connected(0) < Static(1) < OSPF(110)... |
| 5. Lowest Origin Metric --> Lower origin protocol metric wins |
| 6. Lowest Origin Type --> Connected > Static > EBGP > OSPF > IBGP |
| 7. Highest System IP --> Deterministic tie-breaker (Highest IP) |
+---------------------------------------------------------------------------+
Note on Multi-Path (ECMP): If multiple OMP paths tie through all criteria, the router installs up to max-omp-paths (default 4, configurable up to 16) into the forwarding information base for Equal-Cost Multi-Pathing across diverse WAN transports.
2. TLOC Attributes, Colors & Underlay Transport Design
A TLOC (Transport Location) uniquely identifies the physical interface connecting a WAN edge router to an underlay transport provider. An overlay IPsec tunnel is formed between two TLOC endpoints.
The TLOC Key Triplet
A TLOC is universally identified by three mandatory values:
- System IP: A 32-bit unique dotted-decimal identifier (e.g.,
10.255.1.1) assigned to the router. Like a BGP Router ID or loopback, it is non-routable in the underlay transport. - Color: An abstracted text label assigned to the WAN interface representing the transport type (e.g.,
mpls,biz-internet,lte). - Encapsulation: The data plane encapsulation protocol (IPsec [standard default] or GRE).
+-----------------------------------------------------------------------------------+
| TLOC COLOR TAXONOMY & ATTRIBUTES |
+-----------------------------------------------------------------------------------+
| Public Colors (NAT-Friendly): |
| * biz-internet, public-internet, cellular, broadband, 3g, lte, custom1-3 |
| * Behavior: Attempt STUN NAT discovery; expect NAT traversal across Internet. |
+-----------------------------------------------------------------------------------+
| Private Colors (No NAT Expected): |
| * mpls, private1, private2, private3, private4, private5, private6, metro-ethernet|
| * Behavior: Assumes direct private Layer 3 connectivity without NAT. |
+-----------------------------------------------------------------------------------+
| Key Transport Parameters: |
| * 'color restrict': Restricts IPsec tunnel formation exclusively to peer TLOCs |
| sharing the exact same color name (e.g., mpls-to-mpls only). |
| * 'carrier': Distinguishes separate service providers using the same color. |
| * 'tloc-preference': Numeric value (0-4294967295) used by OMP path selection. |
| * 'weight': Controls unequal-cost traffic distribution across active TLOCs. |
+-----------------------------------------------------------------------------------+
+-----------------------+ +-----------------------+
| Branch-101 WAN Edge | | HQ Data Center Edge |
| System IP: 10.255.1.1 | | System IP: 10.255.0.1 |
+-----------------------+ +-----------------------+
| [TLOC: 10.255.1.1 / mpls / ipsec] | [TLOC: 10.255.0.1 / mpls / ipsec]
+===================== Private MPLS Underlay ===============+
| |
| [TLOC: 10.255.1.1 / biz-internet / ipsec] | [TLOC: 10.255.0.1 / biz-internet / ipsec]
+--------------------- Public Internet Underlay ------------+
3. Comprehensive SD-WAN Policy Framework
Cisco Catalyst SD-WAN policies are categorized along two operational axes: Centralized vs. Localized and Control vs. Data.
+-----------------------------------------------------------------------------------+
| SD-WAN POLICY TAXONOMY MATRIX |
+-----------------------------------------------------------------------------------+
| Policy Type | Configuration Point | Execution Point | Key Use Cases |
+--------------------------+---------------------+-----------------+----------------+
| Centralized Control | vManage GUI | vSmart | * Hub-and-Spoke|
| | (Policy Builder) | Controller | Topologies |
| | | | * Custom Meshes|
| | | | * Service Chain|
| | | | * TLOC Steering|
+--------------------------+---------------------+-----------------+----------------+
| Centralized Data | vManage GUI | WAN Edge | * Direct Break-|
| | (Policy Builder) | Data Plane | out (DIA) |
| | | (via vSmart) | * App-Aware Rtg|
| | | | * Flow QoS/Drop|
+--------------------------+---------------------+-----------------+----------------+
| Localized Control | Device Template | WAN Edge | * BGP / OSPF |
| | or CLI | Control Plane | Route-Maps |
| | | | * Prefix Lists |
+--------------------------+---------------------+-----------------+----------------+
| Localized Data | Device Template | WAN Edge | * Traditional |
| | or CLI | Data Plane | Interface QoS|
| | | | * LLQ / Shapers|
| | | | * Queue-Maps |
+--------------------------+---------------------+-----------------+----------------+
Centralized Control Policies: Service Chaining & Topology Control
Centralized Control Policies execute on the vSmart controller before routing updates are distributed to edge routers. They allow administrators to override default full-mesh behavior:
- Hub-and-Spoke Topology: vSmart alters OMP route advertisements so branch edge routers only receive TLOCs belonging to the Hub Data Center, forcing all inter-branch traffic through the Hub.
- Service Chaining (Service Insertion): An edge router at a Hub advertises a Service Route (e.g.,
svc-id 1for a next-generation firewall). vSmart applies a control policy redirecting specific branch traffic to the firewall TLOC before it reaches its final destination.
[Branch A] ----(IPsec Data Tunnel)----> [Hub Firewall (svc-id 1)] ----(Inspected Traffic)----> [Branch B]
^ ^ ^
| | |
+---------------------------------- OMP Advertisements ----------------------------------------+
|
[vSmart Controller]
(Centralized Control Policy Applied)
4. Application-Aware Routing (AAR) & SLA Enforcement
Real-Time Performance Monitoring via BFD
In traditional networks, path selection is static; routing protocols only detect whether a link is completely UP or DOWN. Application-Aware Routing (AAR) introduces dynamic, performance-based traffic steering.
Every WAN edge router runs Bidirectional Forwarding Detection (BFD) packets over every active IPsec data tunnel to peer edges. BFD packets continuously measure three critical SLA metrics:
- Packet Loss: Percentage of dropped BFD packets over a sliding calculation window.
- Latency (Delay): One-way or round-trip transit delay measured in milliseconds (ms).
- Jitter: Delay variance between successive packets measured in milliseconds (ms).
+-----------------------------------------------------------------------------------+
| APPLICATION-AWARE ROUTING WORKFLOW |
+-----------------------------------------------------------------------------------+
[1. Deep Packet Inspection / NBAR2]
* WAN Edge classifies incoming application traffic using NBAR2 / L7 DPI
(e.g., identifying Microsoft Teams, Webex, Zoom, Office 365, Salesforce).
|
v
[2. SLA Class Policy Evaluation]
* Policy checks application traffic against configured SLA Class thresholds:
- Example 'Voice-SLA': Loss <= 1%, Latency <= 150ms, Jitter <= 30ms
|
v
[3. Real-Time BFD Metric Comparison across TLOCs]
* Edge compares current BFD probe metrics for all active underlay transports:
- Transport A (MPLS): Loss=0.1%, Latency=45ms, Jitter=5ms --> [MEETS SLA]
- Transport B (Internet): Loss=3.5%, Latency=120ms, Jitter=45ms --> [VIOLATES SLA]
|
v
[4. Dynamic Flow Steering]
* Edge steers voice application flows to Transport A (MPLS).
* If MPLS degrades (brownout) and Internet improves, traffic dynamically switches.
|
v
[5. Fallback Behavior (If NO Transports Meet SLA)]
* 'fallback best-effort' (Default): Steers traffic to the transport closest to SLA.
* 'strict': Drops application traffic rather than sending over out-of-SLA paths.
+-----------------------------------------------------------------------------------+
5. Policy Verification & Troubleshooting CLI Reference
Essential Verification Commands for OMP, BFD, and AAR
! Verify BFD session state, multiplier, and interval over all IPsec data tunnels
Router# show sdwan bfd sessions
DETECT TX RX
SYSTEM IP SITE ID STATE COLOR SOURCE IP DST IP MULTIPLIER INTERVAL INTERVAL
-----------------------------------------------------------------------------------------------------------------
10.255.0.1 100 up biz-internet 203.0.113.15 198.51.100.2 7 1000 1000
10.255.0.1 100 up mpls 10.1.1.2 10.2.2.2 7 1000 1000
! View real-time loss, latency, and jitter statistics measured by BFD probes
Router# show sdwan app-route statistics
TOTAL PACKET AVERAGE AVERAGE
SRC IP DST IP PKTS LOSS STATUS MEAN-LOSS LATENCY JITTER
--------------------------------------------------------------------------
203.0.113.15 198.51.100.2 12450 0 Valid 0 42 ms 4 ms
10.1.1.2 10.2.2.2 12450 120 Valid 1 18 ms 2 ms
! Inspect OMP routes received from vSmart controller
Router# show sdwan omp routes vpn 10
----------------------------------------------------------------------------------------------------------------------
VPN PREFIX FROM PEER STATUS TLOC IP COLOR ENCAP PREFERENCE METRIC SITE ID
----------------------------------------------------------------------------------------------------------------------
10 172.16.10.0/24 10.255.0.200 C,I,valid 10.255.0.1 biz-internet ipsec 100 0 100
10 172.16.10.0/24 10.255.0.200 C,I,valid 10.255.0.1 mpls ipsec 200 0 100
! Verify centralized data/AAR policies pushed to the edge router from vSmart
Router# show sdwan policy from-vsmart
from-vsmart data-policy AppAware_Branch_Policy
direction all
vpn-list Corp_VPN_10
match
app-list Cisco_Collaboration (Webex, Teams)
action
sla-class Voice_Video_SLA preferred-color mpls
fallback best-effort
Common Policy & Traffic Engineering Faults
| Issue | CLI Diagnosis | Root Cause | Resolution |
|---|---|---|---|
| Tunnels Fail to Form on Private Circuit | show sdwan bfd sessions -> No session / down | Missing color restrict or unmatched color across private WAN. | Add color restrict under interface configuration; ensure matching color names. |
| AAR Traffic Dropped during SLA Outage | show sdwan policy from-vsmart -> action strict | Policy configured with strict fallback dropped traffic when both WAN links suffered brownouts. | Modify policy action to fallback best-effort to preserve connectivity during severe transport degradation. |
| Traffic Blackholing via Hub | show sdwan omp routes -> Missing next-hop | Centralized control policy filtered branch TLOCs without advertising a default route. | Update centralized control policy on vManage to advertise default summary TLOC from Hub. |
| Asymmetric Return Path | show sdwan omp routes -> Unequal preference | Mismatched tloc-preference or preference between redundant edge routers at the same site. | Align OMP route and TLOC preference values in device templates across redundant edge pairs. |
An enterprise SD-WAN deployment has redundant vSmart controllers advertising reachability for prefix 192.168.50.0/24 in VPN 10 to a branch cEdge router. The branch router receives two valid OMP paths:
Which path is selected as the best OMP route, and why?
A network architect wants to enforce a hub-and-spoke routing topology across 500 branch offices and implement service chaining so that all inter-branch traffic is inspected by a centralized next-generation firewall cluster at the primary data center. Which policy mechanism should be configured on vManage to achieve this design?
An enterprise configures an Application-Aware Routing (AAR) policy for real-time video conferencing traffic requiring an SLA of Packet Loss <= 1%, Latency <= 150ms, and Jitter <= 30ms. During a major fiber cut, both the primary MPLS circuit and secondary Internet circuit suffer severe degradation, with MPLS reporting 3% packet loss and Internet reporting 5% packet loss. If the AAR policy is configured with 'fallback best-effort', how does the edge router handle incoming video traffic?
A branch office edge router is connected to a private MPLS underlay on GigabitEthernet0/0/1 with color 'mpls' and to a public broadband Internet underlay on GigabitEthernet0/0/2 with color 'biz-internet'. The security architecture requires that the private MPLS transport NEVER attempt to form IPsec data tunnels to remote edge routers connected via the public Internet. Which configuration parameter must be applied under the GigabitEthernet0/0/1 tunnel interface to enforce this restriction?