5.1 Server & Network Device Virtualization

5.1 Server & Network Device Virtualization

Core Blueprint Focus: Cisco 350-401 ENCOR v1.2 topic 2.1 (Describe device virtualization technologies) requires candidates to master the differences between Type 1 and Type 2 hypervisors, understand virtual machine execution versus containerization (Linux namespaces, cgroups, and layered images), configure and verify Cisco Application Hosting (Docker and IOx) on Cisco Catalyst IOS-XE platforms, and evaluate network virtualization mechanisms such as SR-IOV and device stacking.

Modern enterprise infrastructure relies on virtualization across compute, networking, and storage layers. Virtualization abstracts physical hardware resources, enabling multi-tenancy, rapid workload orchestration, dynamic resource elasticity, and high-density workload consolidation in enterprise data centers, campus backbones, and branch edge routers.

---

1. Hypervisor Architectures: Type 1 (Bare-Metal) vs. Type 2 (Hosted)

A Hypervisor (also known as a Virtual Machine Monitor / VMM) is the software, firmware, or hardware layer responsible for abstracting physical hardware (CPU, Memory, Storage, NICs) and presenting virtualized hardware instances to Virtual Machines (VMs).

+-------------------------------------------------------------------------+
|                    HYPERVISOR ARCHITECTURAL MODELS                      |
|                                                                         |
|         TYPE 1: BARE-METAL                     TYPE 2: HOSTED           |
|   +---------------------------+        +---------------------------+    |
|   |   VM 1   |  VM 2  |  VM 3 |        |   VM 1   |  VM 2  |  VM 3 |    |
|   | (App/OS) |(App/OS)|(App/OS|        | (App/OS) |(App/OS)|(App/OS|    |
|   +---------------------------+        +---------------------------+    |
|   |   Type 1 Hypervisor / VMM |        |   Type 2 Hypervisor (App) |    |
|   | (ESXi / KVM / Hyper-V)    |        | (Workstation / VirtualBox)|    |
|   +---------------------------+        +---------------------------+    |
|   |   Physical Server HW      |        |   Host Operating System   |    |
|   |  (Bare-Metal CPU/RAM/NIC) |        | (Windows / Linux / macOS) |    |
|   +---------------------------+        +---------------------------+    |
|                                        |   Physical Server HW      |    |
|                                        |  (Bare-Metal CPU/RAM/NIC) |    |
|                                        +---------------------------+    |
+-------------------------------------------------------------------------+

Type 1 Hypervisors (Bare-Metal / Native)

• Direct Hardware Execution: Type 1 hypervisors install directly onto physical server bare-metal hardware without any intermediary host operating system. The hypervisor acts as its own specialized, micro-kernel operating system.
• Direct Hardware Control: Device drivers and CPU schedulers run within the hypervisor kernel, enabling sub-millisecond I/O scheduling and direct access to CPU hardware virtualization extensions (Intel VT-x / AMD-V) and Second-Level Address Translation (SLAT / Intel EPT / AMD NPT).
• Enterprise Grade: Delivers maximum I/O throughput, deterministic low latency, and enterprise-grade resilience for production workloads.
• Industry Examples: VMware ESXi, Linux KVM (Kernel-based Virtual Machine), Microsoft Hyper-V (Server Core/Bare-metal), Citrix XenServer.

Type 2 Hypervisors (Hosted)

• Hosted Execution: Type 2 hypervisors install as user-space software applications on top of a pre-existing general-purpose host operating system (such as Microsoft Windows 11, Ubuntu Linux, or macOS).
• Layered Overhead: When a guest VM executes an I/O or CPU instruction, the instruction must pass through the guest OS $\to$ Type 2 hypervisor application $\to$ host OS kernel $\to$ physical hardware. This double context-switching incurs latency, CPU overhead, and jitter.
• Target Use Cases: Software development, network simulation (e.g., Cisco Modeling Labs / CML, EVE-NG on desktop, GNS3), local testing sandboxes, and academic labs.
• Industry Examples: VMware Workstation Pro, VMware Fusion, Oracle VM VirtualBox, Parallels Desktop.

Detailed Architectural Comparison

---

2. Virtual Machines vs. Containers (Docker & OCI)

While Virtual Machines virtualize the physical hardware stack (running independent guest operating systems with separate kernels), Containers virtualize at the Operating System level, allowing multiple isolated applications to share the same underlying host OS kernel.

+-------------------------------------------------------------------------+
|                    VIRTUAL MACHINES VS. CONTAINERS                      |
|                                                                         |
|         VIRTUAL MACHINES                         CONTAINERS             |
|   +---------------------------+        +---------------------------+    |
|   | App A  |  App B  |  App C |        | App A  |  App B  |  App C |    |
|   +--------+---------+--------+        +--------+---------+--------+    |
|   | Libs/Bin|Libs/Bin|Libs/Bin|        | Libs/Bin|Libs/Bin|Libs/Bin|    |
|   +--------+---------+--------+        +---------------------------+    |
|   | GuestOS| GuestOS | GuestOS|        | Container Runtime Engine  |    |
|   | (Linux)|(Windows)| (Linux)|        | (Docker / containerd / CRI|    |
|   +---------------------------+        +---------------------------+    |
|   | Type 1/2 Hypervisor (VMM) |        | Host OS Kernel (Linux)    |    |
|   +---------------------------+        | (Namespaces & Cgroups)    |    |
|   | Physical Server Hardware  |        +---------------------------+    |
|   | (CPU / RAM / Disk / NIC)  |        | Physical Server Hardware  |    |
|   +---------------------------+        +---------------------------+    |
+-------------------------------------------------------------------------+

Linux Kernel Primitives Powering Containers

Containers are not full operating systems; they are standard Linux processes isolated by two fundamental Linux kernel subsystems:

1. Linux Namespaces (Process Isolation & Boundary):
   • Namespaces partition global system resources so that a container process sees only its own isolated environment:
     • PID Namespace: Isolates process IDs (the container entrypoint process sees itself as PID 1).
     • NET Namespace: Provides an independent network stack (private IP addresses, routing tables, iptables/nftables, and virtual ethernet veth interfaces).
     • MNT Namespace: Isolates filesystem mount points, providing a private root filesystem.
     • IPC Namespace: Isolates Inter-Process Communication (shared memory segments, POSIX message queues).
     • UTS Namespace: Isolates hostname and NIS domain name.
     • USER Namespace: Maps the container root user (UID 0) to an unprivileged user ID on the host OS for enhanced security.
2. Control Groups (cgroups / cgroups v2 - Resource Quotas):
   • Enforces strict hardware resource limits, quotas, and prioritization across container processes:
     • CPU Shares & Quotas: Limits maximum CPU cores or CPU cycles allocated to a container.
     • Memory Limits: Restricts physical RAM and swap usage (triggers the Linux OOM-killer if exceeded).
     • Block I/O (blkio): Throttles disk read/write throughput and IOPS.
     • Network Bandwidth: Prioritizes and rate-limits network socket egress traffic.
3. Union Filesystems & Layered Images (OverlayFS / CoW):
   • Container images are built using immutable read-only layers with a lightweight thin read-write layer on top using Copy-on-Write (CoW) mechanics, minimizing storage consumption and enabling instant startup.

Detailed Comparison: Containers vs. Virtual Machines

---

3. Cisco Catalyst IOS-XE Application Hosting (IOx & Docker)

Modern enterprise switching and routing platforms (such as Cisco Catalyst 9300, 9400, 9500, 9600 switches and Catalyst 8200/8300/8500 edge routers) are powered by Cisco IOS-XE. Underneath the IOS-XE control plane runs a hardened Linux operating system (CentOS/Yocto distribution).

Through the Cisco IOx Application Hosting framework, network administrators can deploy, execute, and manage custom third-party applications (e.g., ThousandEyes Enterprise Agents, Cisco Cyber Vision sensors, Python network automation scripts, Wireshark packet capture tools, Prometheus node exporters) directly on network devices without requiring external server hardware.

+-------------------------------------------------------------------------+
|               CISCO CATALYST IOS-XE APPLICATION HOSTING                 |
|                                                                         |
|   +-----------------------------------------------------------------+   |
|   |                   CISCO CATALYST 9000 HARDWARE                  |   |
|   |   (Multi-Core x86 CPU, 16G-32G RAM, Internal SSD / USB 3.0)     |   |
|   +-----------------------------------------------------------------+   |
|   |                     HOST LINUX KERNEL                           |   |
|   |   +--------------------------+   +--------------------------+   |   |
|   |   |  Cisco IOS-XE Process    |   |  Cisco IOx Daemon (caf)  |   |   |
|   |   |  - Control Plane Routing |   |  - Docker Container Eng  |   |   |
|   |   |  - Hardware ASIC Driver  |   |  - KVM Hypervisor Engine |   |   |
|   |   +------------|-------------+   +------------|-------------+   |   |
|   +----------------|------------------------------|-----------------+   |
|                    |                              |                     |
|                    v                              v                     |
|   +----------------------------------+   +--------------------------+   |
|   | Switch ASIC / Hardware Forwarding|   | Hosted Docker Containers |   |
|   | - Wire-rate CEF FIB              |   | - ThousandEyes Agent     |   |
|   | - Front-panel 10G/25G/100G Ports |   | - Custom Python App      |   |
|   +----------------|-----------------+   +------------|-------------+   |
|                    |                                  |                 |
|                    +===== AppGigabitEthernet 1/0/1 ===+                 |
|                           (Internal HW Bridge Port)                     |
+-------------------------------------------------------------------------+

Application Types Supported in Cisco IOx

1. Docker Containers (Native OCI): Packaged Docker .tar images executing natively under the IOx Docker runtime engine.
2. KVM Virtual Machines: Complete guest operating system images (.qcow2 or .raw) running atop the embedded Linux KVM hypervisor.
3. LXC (Linux Containers): Traditional POSIX container format.

Application Hosting Networking Architecture

To connect hosted applications to the enterprise network, Cisco IOS-XE provides two primary interface mechanisms:

• AppGigabitEthernet (AppGigEth):
  • A dedicated internal hardware interface present on Cisco Catalyst 9000 switches (e.g., AppGigabitEthernet 1/0/1).
  • Connects the internal switch fabric directly to the application hosting CPU complex.
  • Operates as a standard Layer 2 switchport, configurable as an Access Port (single VLAN) or an 802.1Q Trunk Port (multiple VLANs passed directly into the container).
• VirtualPortGroup (VPG):
  • A virtual Layer 3 routed interface commonly used on Cisco Catalyst 8000 and ISR 4000 series routers (e.g., interface VirtualPortGroup0).
  • Operates as an internal routed default gateway for hosted applications. Traffic from the container is routed via the VPG into specific VRFs or the global routing table, often utilizing Network Address Translation (NAT).

Resource Allocation & Governance

Cisco IOS-XE enforces strict CPU and memory governance using Linux cgroups to guarantee that hosted applications never starve the IOS-XE control plane or impact packet forwarding:

• CPU Quota: Configured as a percentage of total CPU cycles (e.g., 10% to 50%) or dedicated CPU cores.
• Memory Limit: Maximum physical DRAM allocated to the container (e.g., 512 MB to 4096 MB).
• Storage Quota: Dedicated persistent disk space on internal flash (bootflash:), external USB 3.0 SSD, or SATA M.2 SSD (harddisk:).

---

4. Cisco IOS-XE Application Hosting Configuration & Verification

The following step-by-step example illustrates enabling IOx, configuring the dedicated internal AppGigabitEthernet trunk, provisioning a ThousandEyes Docker container with static IP settings, and managing the application lifecycle via Cisco IOS-XE CLI.

! Step 1: Enable the Cisco IOx application hosting infrastructure
Switch(config)# iox

! Step 2: Configure the internal AppGigabitEthernet interface as an 802.1Q Trunk
Switch(config)# interface AppGigabitEthernet 1/0/1
Switch(config-if)# description Internal Application Hosting Bridge
Switch(config-if)# switchport mode trunk
Switch(config-if)# switchport trunk allowed vlan 10,20,100
Switch(config-if)# no shutdown
Switch(config-if)# exit

! Step 3: Configure the Application Hosting container profile
Switch(config)# app-hosting appid thousandeyes_agent
Switch(config-app-hosting)# app-vnic AppGigabitEthernet trunk
Switch(config-app-hosting-vnic)# vlan 100 guest-interface 0
Switch(config-app-hosting-vnic-vlan)# guest-ipaddress 10.100.1.50 netmask 255.255.255.0
Switch(config-app-hosting-vnic-vlan)# exit
Switch(config-app-hosting)# app-default-gateway 10.100.1.1 guest-interface 0
Switch(config-app-hosting)# name-server0 10.1.1.10
Switch(config-app-hosting)# app-resource docker
Switch(config-app-hosting-docker)# run-opts 1 "-e TEAGENT_ACCOUNT_TOKEN=abc123xyz456"
Switch(config-app-hosting-docker)# exit
Switch(config-app-hosting)# app-resource profile custom
Switch(config-app-resource)# cpu 1500
Switch(config-app-resource)# memory 1024
Switch(config-app-resource)# persist-disk 2048
Switch(config-app-resource)# exit
Switch(config-app-hosting)# exit

Application Hosting Lifecycle Management Commands

! Install the Docker container package from flash storage
Switch# app-hosting install appid thousandeyes_agent package bootflash:thousandeyes-enterprise-agent-4.3.0.tar
Installing package 'bootflash:thousandeyes-enterprise-agent-4.3.0.tar' for appid 'thousandeyes_agent'...
Successfully installed package for appid 'thousandeyes_agent'.

! Activate the application (allocates CPU, memory, and vNIC resources)
Switch# app-hosting activate appid thousandeyes_agent
Activating app 'thousandeyes_agent'...
Successfully activated app 'thousandeyes_agent'.

! Start the application container
Switch# app-hosting start appid thousandeyes_agent
Starting app 'thousandeyes_agent'...
Successfully started app 'thousandeyes_agent'.

Verification and Monitoring Commands

! Display summary status of all hosted applications
Switch# show app-hosting list
App id                                   State
------------------------------------------------------
thousandeyes_agent                       RUNNING

! Display comprehensive hardware allocation and networking details
Switch# show app-hosting detail
App id                 : thousandeyes_agent
Owner                  : iox
State                  : RUNNING
Application
  Type                 : docker
  Name                 : ThousandEyes Enterprise Agent
  Version              : 4.3.0
  Description          : Enterprise network synthetic monitoring agent
  Path                 : bootflash:thousandeyes-enterprise-agent-4.3.0.tar
Activated profile name : custom

Resource reservation
  Memory               : 1024 MB
  Disk                 : 2048 MB
  CPU                  : 1500 units (1.5 Cores)
  VCPU                 : 2

Attached devices
  Type              Name               Alias
  ---------------------------------------------
  NIC               AppGigEth1/0/1     guest-interface0
    VLAN ID         : 100
    Guest IP Address: 10.100.1.50
    MAC Address     : 52:54:dd:11:22:33

! Display global IOx framework status
Switch# show app-hosting infra
IOX version: 2.7.0.0
App signatures: Disabled
Internal working directory: /vol/usb1/iox

CAF status: Running
HA status: Not Supported

---

5. Network Device Virtualization & Hardware Acceleration

In addition to server virtualization and application hosting, enterprise architectures leverage device-level and data path virtualization to optimize throughput, redundancy, and management efficiency.

Single Root I/O Virtualization (SR-IOV)

In standard virtual switching, all network packets generated by a VM traverse the hypervisor virtual switch software driver, consuming CPU cycles and adding jitter. Single Root I/O Virtualization (SR-IOV) allows a single physical PCIe network adapter to natively partition itself into multiple virtual PCIe devices:

• Physical Function (PF): The primary PCIe function containing full device management capabilities, used by the hypervisor to allocate and manage Virtual Functions.
• Virtual Function (VF): A lightweight PCIe function containing dedicated hardware queues and DMA channels, assigned directly to a VM's PCI bus via IOMMU (Intel VT-d / AMD-Vi).
• Benefits & Trade-offs: Delivers line-rate 100G+ throughput and sub-microsecond latency, but bypasses the hypervisor software switch, disabling hypervisor-level vMotion, software ACLs, and NetFlow monitoring.

Network Device Clustering & Multi-Instance Virtualization

• Many-to-One Virtualization (Clustering / Stacking):
  • Cisco StackWise-480 / StackWise-1T: Physically cables up to 9 Catalyst switches into a single logical switch with a single management IP and shared control plane.
  • Cisco StackWise Virtual (SVL): Combines two modular/fixed core switches (e.g., Catalyst 9500/9600) over 40G/100G fiber links into a single logical entity with Dual-Active Detection (DAD) to eliminate Spanning Tree loops.
• One-to-Many Virtualization (Multi-Instance / Slicing):
  • Cisco Nexus Virtual Device Contexts (VDC): Partitions a single physical Cisco Nexus 7000/7700 chassis into multiple independent logical switches, each with dedicated physical interfaces, independent control plane processes, and isolated failure domains.