5.3 GRE & IPsec Tunnel Virtualization Mechanics
Key Takeaways
- Generic Routing Encapsulation (GRE / RFC 2784) is an unencrypted, stateless tunneling protocol (IP protocol 47) that encapsulates multi-protocol payload packets inside an outer IP header, enabling dynamic routing protocols (OSPF, EIGRP, BGP) and multicast traffic across unicast IP networks.
- A standard GRE/IP tunnel introduces exactly 24 bytes of overhead (20-byte outer IPv4 header + 4-byte GRE header), reducing the default 1500-byte interface MTU to an IP MTU of 1476 bytes and requiring TCP MSS clamping (`ip tcp adjust-mss 1436`) to prevent Path MTU Discovery black holes.
- IPsec operates in two modes: Transport Mode (encrypts only the payload and preserves the original IP header, adding 36-56 bytes of overhead) and Tunnel Mode (encapsulates and encrypts the entire original IP packet within a new outer IP header, adding 50-70+ bytes of overhead).
- IPsec Security Protocols comprise Encapsulating Security Payload (ESP / IP protocol 50), which provides data confidentiality (encryption), integrity, data origin authentication, and anti-replay, and Authentication Header (AH / IP protocol 51), which provides integrity and authentication without encryption and fails when traversing NAT.
- Internet Key Exchange version 2 (IKEv2 / RFC 7296) streamlines Phase 1 and Phase 2 negotiations into 4 initial messages (IKE_SA_INIT and IKE_AUTH), natively integrates NAT-Traversal (UDP port 4500), supports asymmetric authentication methods, and provides built-in Dead Peer Detection (DPD) and DoS cookie protection.
5.3 GRE & IPsec Tunnel Virtualization Mechanics
Core Blueprint Focus: Cisco 350-401 ENCOR v1.2 topic 2.2.b (GRE and IPsec tunneling) requires candidates to configure and verify GRE tunnels, analyze encapsulation overhead and MTU/MSS clamping, differentiate between IPsec Transport and Tunnel modes, evaluate ESP versus AH security protocols, and implement modern IKEv2 site-to-site Virtual Tunnel Interfaces (VTI) on Cisco IOS-XE platforms.
Enterprise WANs and cloud architectures require logical overlay networks to interconnect geographically dispersed sites across untrusted or multi-provider transport backbones. Tunnel virtualization technologies decouple the overlay logical topology from the underlying physical transport, enabling dynamic routing, multicast distribution, and end-to-end cryptographic protection.
1. Generic Routing Encapsulation (GRE / RFC 2784)
Generic Routing Encapsulation (GRE) is a lightweight, stateless tunneling protocol developed by Cisco and standardized in RFC 2784 and RFC 1701. GRE encapsulates a wide variety of network-layer payload protocols (such as IPv4, IPv6, and MPLS) inside an outer IP transport header.
+-------------------------------------------------------------------------+
| GRE PACKET ENCAPSULATION ANATOMY |
| |
| |<---------------------- Outer IP Packet (1500 B) ------------------>| |
| | | |
| +-------------------+---------------+-------------------------------+ |
| | Outer IP Header | GRE Header | Original Passenger IP | |
| | (Delivery) | (RFC 2784) | Packet (Inner IP + Data) | |
| | 20 Bytes | 4 Bytes | 1476 Bytes | |
| +-------------------+---------------+-------------------------------+ |
| | Protocol: 47 | EtherType: | Src/Dst: 10.1.1.1 -> 10.2.2.2 | |
| | Src: 198.51.100.1 | 0x0800 (IPv4) | Payload: TCP/UDP Data | |
| | Dst: 203.0.113.1 | | | |
| +-------------------+---------------+-------------------------------+ |
+-------------------------------------------------------------------------+
Key Technical Characteristics of GRE
- IP Protocol Number 47: GRE packets operate directly on top of IP using IP protocol number 47.
- Multicast and Broadcast Transport: Unlike native IPsec tunnel mode (which cannot encapsulate multicast without specialized extensions), GRE natively encapsulates IP multicast and broadcast packets. This allows dynamic routing protocols (e.g., OSPF
224.0.0.5/224.0.0.6, EIGRP224.0.0.10) to establish neighbor adjacencies across the tunnel. - Stateless Operation: GRE tunnels have no keepalive state machine by default. An unconfigured GRE tunnel interface remains in
up/upline protocol state as long as a valid route exists to the tunnel destination IP address in the local routing table. Enablingkeepalivesends periodic GRE keepalives to detect remote endpoint failure. - No Native Encryption: GRE provides zero data confidentiality. Payloads are transmitted in cleartext unless encapsulated inside an IPsec security association (GRE over IPsec).
Standard GRE Header Fields (RFC 2784)
- C (Checksum Present - Bit 0): Set to 1 if a Checksum field is present.
- Flags & Version (Bits 1-15): Must be set to zero in RFC 2784 compliance (Version = 0).
- Protocol Type (Bits 16-31, 2 Bytes): Specifies the EtherType of the encapsulated passenger payload (e.g.,
0x0800for IPv4,0x86DDfor IPv6).
2. Tunnel Overhead, MTU Fragmentation & TCP MSS Clamping
When encapsulating packets inside GRE or IPsec tunnels, the added headers increase the total packet size. If the resulting packet exceeds the physical path Maximum Transmission Unit (MTU), fragmentation occurs, leading to performance degradation, retransmissions, or dropped packets.
+-------------------------------------------------------------------------+
| GRE TUNNEL MTU & TCP MSS CALCULATION |
| |
| |<------------------- Physical Link MTU: 1500 Bytes ---------------->| |
| | | |
| +------------------+--------------+---------------------------------+ |
| | Outer IPv4 (20B) | GRE (4B) | Inner IP Payload (1476B) | |
| +------------------+--------------+---------------------------------+ |
| | <--- GRE Overhead: 24B ---> | | |
| | |
| |<------ IP MTU: 1476 Bytes ----->| |
| | |
| +------------+-----------+--------+ |
| | Inner IPv4 | TCP | TCP | |
| | Header(20B)|Header(20B)| Payload| |
| +------------+-----------+--------+ |
| | <-- Header Overhead: 40B --> | |
| | |
| |<----- TCP MSS: 1436 Bytes ----->| |
+-------------------------------------------------------------------------+
Mathematical Calculations
-
Standard GRE Overhead:
-
Tunnel Interface IP MTU:
-
TCP Maximum Segment Size (MSS) Clamping:
Path MTU Discovery (PMTUD) & The "Black Hole" Problem
When an endpoint sends a packet with the Don't Fragment (DF) bit set in the IP header and the packet size exceeds an intermediate router's MTU:
- The router drops the packet.
- The router transmits an ICMP Type 3, Code 4 message (Destination Unreachable: Fragmentation Needed and DF Set) containing the next-hop MTU back to the sender.
- The Black Hole Problem: In many enterprise WANs and internet connections, intermediate firewalls or security gateways drop all inbound ICMP messages. As a result, the sender never receives the ICMP error, never reduces its packet size, and continues retransmitting dropped packets indefinitely.
- The Solution (TCP MSS Clamping): By applying
ip tcp adjust-mss 1436to the tunnel interface, the router intercepts TCP SYN and SYN-ACK packets during the initial 3-way handshake and rewrites the advertised MSS option value to1436bytes. This forces both client and server to limit their TCP segments, completely eliminating PMTUD black holes and fragmentation.
3. IPsec Security Architecture: Transport vs. Tunnel Mode
IP Security (IPsec) is an open architectural framework (RFC 4301) providing network-layer security services: Confidentiality (encryption), Data Integrity, Data Origin Authentication, and Anti-Replay Protection.
+-------------------------------------------------------------------------+
| IPSEC TRANSPORT VS. TUNNEL MODE |
| |
| CLEARTEXT IP PACKET: |
| +-----------------------+-------------------------------------------+ |
| | Orig IP Header | TCP / Payload | |
| +-----------------------+-------------------------------------------+ |
| |
| IPSEC TRANSPORT MODE (Host-to-Host / GRE-over-IPsec / VTI): |
| +---------------+------------+----+-------------------+-----+-------+ |
| | Orig IP Header| ESP Header | IV | TCP / Payload | ESP | ESP | |
| | (Preserved) | (Prot 50) | | (ENCRYPTED) |Trlr | Auth | |
| +---------------+------------+----+-------------------+-----+-------+ |
| |<-------------- ENCRYPTED ---------->| | |
| |<----------------- AUTHENTICATED ----------------->| |
| |
| IPSEC TUNNEL MODE (Site-to-Site LAN-to-LAN VPN): |
| +---------------+------------+----+-------------------+-----+-------+ |
| | New IP Header | ESP Header | IV | Original Packet | ESP | ESP | |
| | (New Gateway) | (Prot 50) | | (IP Header + Data)|Trlr | Auth | |
| +---------------+------------+----+-------------------+-----+-------+ |
| |<-------------- ENCRYPTED ---------->| | |
| |<----------------- AUTHENTICATED ----------------->| |
+-------------------------------------------------------------------------+
Transport Mode vs. Tunnel Mode Comparison
| Technical Attribute | IPsec Transport Mode | IPsec Tunnel Mode |
|---|---|---|
| Header Preservation | Preserves original IP header; inserts ESP after IP header | Encapsulates entire original IP packet inside a New Outer IP header |
| Payload Protection | Encrypts upper-layer payload (Transport/Application) | Encrypts entire original packet (Inner IP header + Payload) |
| Addressing Visibility | Original source/destination IP addresses are exposed on wire | Original source/destination IP addresses are completely hidden |
| Protocol Overhead | Lower overhead (typically 36–56 bytes) | Higher overhead (typically 50–74 bytes due to extra IP header) |
| Typical Deployment | End-to-end host communication; GRE-over-IPsec; VTI | Traditional Site-to-Site Gateway-to-Gateway VPNs |
4. IPsec Security Protocols: ESP (Protocol 50) vs. AH (Protocol 51)
IPsec provides two distinct Layer 3 security protocols:
+-------------------------------------------------------------------------+
| ESP VS. AH PROTOCOL MATRIX |
| |
| SECURITY SERVICE ESP (Protocol 50) AH (Protocol 51) |
| ------------------------------------------------------------------- |
| Confidentiality (Encryption) YES (AES-GCM/CBC) NO (Cleartext) |
| Data Integrity (HMAC/Hashing) YES (SHA-256/512) YES (SHA-256/512) |
| Data Origin Authentication YES (Pre-Shared/Cert)YES (Pre-Shared) |
| Anti-Replay Protection YES (Seq Numbers) YES (Seq Numbers) |
| NAT Traversal (NAT-T UDP 4500) YES NO (NAT Incompatible)
+-------------------------------------------------------------------------+
Encapsulating Security Payload (ESP - IP Protocol 50)
- Complete Security Suite: ESP provides confidentiality via symmetric encryption algorithms (AES-128, AES-256, AES-GCM), integrity and authentication via HMAC hashes (SHA-256, SHA-384, SHA-512), and anti-replay protection via 32-bit/64-bit sequence numbers.
- NAT-Traversal (NAT-T / RFC 3948): When a router or firewall detects a Network Address Translation (NAT/PAT) device along the path, standard ESP fails because ESP contains no Layer 4 port numbers for PAT translation. NAT-T solves this by encapsulating ESP inside UDP port 4500, allowing encrypted packets to traverse PAT gateways seamlessly.
Authentication Header (AH - IP Protocol 51)
- Integrity Without Confidentiality: AH calculates an Integrity Check Value (ICV) across the payload and the outer IP header (excluding mutable fields like TTL and TOS).
- Zero Encryption: All data is transmitted in cleartext.
- NAT Incompatibility: When a NAT router translates IP addresses or ports, the altered IP header invalidates the AH ICV hash at the destination peer, causing immediate packet drops. AH cannot operate across NAT.
5. Key Exchange Protocols: IKEv1 vs. IKEv2 Architecture
Internet Key Exchange (IKE) dynamically authenticates peers, negotiates cryptographic algorithms, and manages Security Associations (SAs) using UDP port 500 (and UDP port 4500 for NAT-T).
+-------------------------------------------------------------------------+
| IKEv2 4-MESSAGE NEGOTIATION |
| |
| INITIATOR RESPONDER |
| |
| [1. IKE_SA_INIT Request] ============================> |
| - Security Proposal (Encryption, Hash, DH Group) |
| - Diffie-Hellman Public Value (g^x) & Nonce (Ni) |
| |
| <=========================== [2. Response] |
| - Chosen Proposal & DH Value (g^y) |
| - Nonce (Nr) & Optional NAT Detection |
| |
| ================== ENCRYPTED CHANNEL ESTABLISHED =================== |
| |
| [3. IKE_AUTH Request] ============================> |
| - Peer Identity (IDi: IP/FQDN/Cert) |
| - Authentication (PSK Hash or Digital Signature) |
| - First Child SA Proposal (Traffic Selectors / TSi, TSr) |
| |
| <=========================== [4. Response] |
| - Responder Identity (IDr) & Auth Data |
| - First Child SA Accepted Proposal |
+-------------------------------------------------------------------------+
IKEv2 vs. IKEv1 Architectural Improvements
| Technical Attribute | IKEv1 (RFC 2409) | IKEv2 (RFC 7296) |
|---|---|---|
| Message Exchanges | Phase 1: 6 packets (Main Mode) or 3 (Aggressive) + Phase 2: 3 packets (Quick Mode) = 6 to 9 packets | 4 initial packets (IKE_SA_INIT + IKE_AUTH) to establish both IKE SA and first Child SA |
| NAT-Traversal Support | Vendor-specific draft extension; requires post-discovery renegotiation | Natively integrated in IKE_SA_INIT; automatic port float to UDP 4500 |
| Authentication Flexibility | Symmetric only (both peers must use PSK or both use Certs) | Asymmetric authentication (e.g., PSK on one peer, RSA certificate on other, or EAP) |
| Dead Peer Detection (DPD) | Optional proprietary keepalive extension | Native standard keepalive mechanism |
| DoS / Anti-Spoofing | Vulnerable to half-open state exhaustion | Anti-DoS Cookie Challenge mechanism under attack conditions |
| Bandwidth Efficiency | Higher control-plane overhead | Significantly reduced round-trip latency and CPU overhead |
6. Cisco IOS-XE GRE & IKEv2 SVTI Configuration & Verification
The following configuration illustrates establishing an IKEv2 Static Virtual Tunnel Interface (SVTI) over IPv4 with AES-256-GCM encryption and TCP MSS clamping on Cisco IOS-XE.
! Step 1: Configure IKEv2 Proposal and Policy
crypto ikev2 proposal IKEV2-PROP-ENTERPRISE
encryption aes-gcm-256
prf sha384
group 19 20
!
crypto ikev2 policy IKEV2-POL-ENTERPRISE
proposal IKEV2-PROP-ENTERPRISE
!
! Step 2: Configure IKEv2 Keyring (Pre-Shared Keys)
crypto ikev2 keyring IKEV2-KEYRING-BRANCH
peer BRANCH-ROUTER-1
address 203.0.113.2
pre-shared-key CiscoSecureKey2026!
!
! Step 3: Configure IKEv2 Profile
crypto ikev2 profile IKEV2-PROF-BRANCH
match identity remote address 203.0.113.2 255.255.255.255
identity local address 198.51.100.1
authentication remote pre-share
authentication local pre-share
keyring local IKEV2-KEYRING-BRANCH
dpd 10 2 on-demand
!
! Step 4: Configure IPsec Transform Set and IPsec Profile
crypto ipsec transform-set TS-AES-GCM esp-gcm 256
mode transport
!
crypto ipsec profile IPSEC-PROF-VTI
set transform-set TS-AES-GCM
set ikev2-profile IKEV2-PROF-BRANCH
!
! Step 5: Configure the Static Virtual Tunnel Interface (SVTI)
interface Tunnel100
description Secure Site-to-Site IKEv2 VTI to Branch 1
ip address 10.254.1.1 255.255.255.252
ip mtu 1400
ip tcp adjust-mss 1360
tunnel source GigabitEthernet0/0/0
tunnel destination 203.0.113.2
tunnel mode ipsec ipv4
tunnel protection ipsec profile IPSEC-PROF-VTI
no shutdown
Verification and Diagnostic Commands
! Verify IKEv2 Phase 1 Security Association
Router# show crypto ikev2 sa
IPv4 Crypto IKEv2 SA
Tunnel-id Local Remote fvrf/ivrf Status
1 198.51.100.1/500 203.0.113.2/500 none/none READY
Encr: AES-GCM, keysize: 256, PRF: SHA384, Hash: None, DH Grp: 19, Auth: PRE-SHARED
Life/Active Time: 86400/1245 sec
! Verify IPsec Phase 2 Child Security Association (Encaps/Decaps counters)
Router# show crypto ipsec sa interface Tunnel100
interface: Tunnel100
Crypto map tag: Tunnel100-head-0, local addr 198.51.100.1
protected vrf: (none)
local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
current_peer 203.0.113.2 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 48291, #pkts encrypt: 48291, #pkts digest: 0
#pkts decaps: 48280, #pkts decrypt: 48280, #pkts verify: 0
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts no encapsulation: 0, #pkts encapsulation failed: 0
#send errors 0, #recv errors 0
local crypto endpt.: 198.51.100.1, remote crypto endpt.: 203.0.113.2
plaintext mtu 1400, path mtu 1500, ip mtu 1400, ip mtu idb GigabitEthernet0/0/0
current outbound spi: 0x8F4A12BC(2403996348)
inbound esp sas:
spi: 0x3C910B8D (1016138637)
transform: esp-gcm 256 , in use settings ={\Transport, }
in use sas: conn_id=2001, flow_id=1
sa timing: remaining key lifetime (sec): 3420
IV size: 8 bytes
replay detection support: Y
Status: ACTIVE(sa:ready)
A network engineer configures a point-to-point standard GRE tunnel across an Ethernet physical infrastructure with a standard MTU of 1500 bytes. To prevent packet fragmentation and eliminate Path MTU Discovery (PMTUD) black holes for TCP traffic traversing the tunnel, what IP MTU and TCP MSS values must be configured on the tunnel interface?
Why does the Authentication Header (AH / IP protocol 51) fail when traversing a Network Address Translation (NAT/PAT) boundary, while Encapsulating Security Payload (ESP) with NAT-Traversal succeeds?
When contrasting IKEv1 and IKEv2 protocols for site-to-site IPsec VPN deployments, what is a primary operational advantage of IKEv2?
A network architect is designing an encrypted tunnel between two corporate data centers over a public transit provider. The design specifies configuring a Virtual Tunnel Interface (VTI) with IPsec protection. Why should IPsec Transport Mode be configured instead of IPsec Tunnel Mode on the transform set?