6.2 Virtual Networking, Storage, and VM Migration

Virtual Networking, Storage, and VM Migration

Core Infrastructure Rule: Virtual machine mobility, network isolation, and storage resilience require strict architectural alignment between physical hardware and software-defined abstractions. Live migration mechanisms demand dedicated low-latency network paths, shared cluster storage fabrics, and uniform processor instruction sets to transfer active execution states without dropping network sessions or corrupting in-flight transactions.

Virtualization abstracts not only compute and memory, but the entire data center network and storage fabric. Modern systems administrators must design virtual switching fabrics that enforce granular VLAN segmentation, configure hardware-assisted I/O virtualization for demanding throughput workloads, and execute zero-downtime workload migrations across clustered server environments.

---

Virtual Networking Components: Switches, Adapters, and Port Groups

Virtual networking replicates physical Layer 2 Ethernet switching entirely in software inside the hypervisor kernel, enabling virtual machines on the same host or across hosts to communicate securely.

+-------------------------------------------------------------------------+
|                       HYPERVISOR HOST NETWORKING                        |
|                                                                         |
|  +--------------------+                    +--------------------+       |
|  |    GUEST VM 1      |                    |    GUEST VM 2      |       |
|  |  [vNIC: vmxnet3]   |                    |  [vNIC: virtio]    |       |
|  +---------+----------+                    +---------+----------+       |
|            |                                         |                  |
|  +---------+-----------------------------------------+---------------+  |
|  |                  VIRTUAL SWITCH (vSwitch / vDS)                   |  |
|  |                                                                   |  |
|  |   +--------------------------+     +--------------------------+   |  |
|  |   | Port Group: Production   |     | Port Group: Database     |   |  |
|  |   | VLAN ID: 100 (VST Mode)  |     | VLAN ID: 200 (VST Mode)  |   |  |
|  |   +------------+-------------+     +------------+-------------+   |  |
|  |                |                                |                 |  |
|  |   +------------+--------------------------------+-------------+   |  |
|  |   |           Uplink Port Group / NIC Team (LACP / IP Hash)   |   |  |
|  +---+---------------------------+-------------------------------+---+  |
|                                  |                                      |
|                       +----------+----------+                           |
|                       | Physical NICs (pNIC)|                           |
|                       | (vmnic0 / vmnic1)   |                           |
+-----------------------+----------+----------+---------------------------+
                                   | (802.1Q Trunked Copper/Fiber)
                                   v
                    [PHYSICAL TOP-OF-RACK SWITCH]

Standard Virtual Switches (vSwitch) vs. Distributed Virtual Switches (vDS)

Hypervisors provide two structural tiers of virtual switching:

1. Standard Virtual Switch (vSwitch / Host-Local Switch):
   • Operates strictly within the boundary of a single physical hypervisor host.
   • Every hypervisor node must be configured independently. In a 32-node cluster, creating a new VLAN or adjusting MTU settings requires making 32 identical manual or scripted configurations.
   • Configuration drift between hosts can disrupt live migration if port group names, security parameters, or VLAN IDs do not match identically.
2. Distributed Virtual Switch (vDS / Distributed Switch):
   • Spans across an entire cluster of hypervisor hosts (up to hundreds of nodes), managed centrally through a unified management plane (such as VMware vCenter Server).
   • Decoupled Architecture: Separates the Control Plane (managed centrally via the cluster management software) from the Data Plane (executed locally in each host's kernel). If the central management server goes offline, data switching continues uninterrupted.
   • Advanced Capabilities:
     • Port State Persistence (Network vMotion): Tracks virtual port connection statistics, filtering states, and security counters seamlessly as a VM live-migrates from Host A to Host B.
     • Private VLANs (PVLANs): Enables Promiscuous, Isolated, and Community VLAN segmentation inside virtual port groups.
     • Traffic Shaping: Implements bidirectional policing and shaping (Average Bandwidth, Peak Bandwidth, Burst Size) for both ingress and egress traffic.
     • Network Telemetry: Supports enterprise packet monitoring protocols including NetFlow, IPFIX, and Port Mirroring (Encapsulated Remote SPAN / ERSPAN).

Virtual Network Interface Cards (vNICs)

A virtual machine connects to a virtual switch port group via an emulated or paravirtualized network adapter:

• Emulated Network Adapters (e.g., Intel E1000 / E1000e): Emulates legacy physical hardware in software. The guest OS utilizes standard native drivers. However, every packet transmitted triggers hypervisor hardware traps and CPU register emulation, resulting in high CPU overhead and restricted packet-per-second (PPS) performance.
• Paravirtualized Network Adapters (VMware vmxnet3, Linux/KVM virtio-net): High-performance virtualization-aware adapters designed specifically for virtual environments. They bypass legacy hardware register emulation, providing:
  • Zero-copy packet ring buffers.
  • Hardware offload support: TCP Segmentation Offload (TSO), Large Receive Offload (LRO), Receive Side Scaling (RSS), and Checksum Offload.
  • Multiqueue support to distribute packet processing across multiple vCPUs.
  • Requirement: Requires hypervisor guest integration tools (VMware Tools, VirtIO drivers) installed in the guest OS.

Port Groups and Security Policies

A Port Group is a logical template of virtual ports on a virtual switch that enforces uniform network configuration, QoS, VLAN tagging, and Layer 2 security policies:

• Promiscuous Mode: When set to Reject, a vNIC receives only packets addressed directly to its own MAC address or broadcast/multicast packets. When set to Accept, the vNIC can capture all traffic traversing the port group (required for virtual intrusion detection systems, packet sniffers, or nested virtualization).
• MAC Address Changes: Controls whether a VM can receive traffic after altering its guest OS MAC address from the original MAC address assigned in the VM configuration (.vmx). Setting this to Reject drops incoming frames if the destination MAC does not match the provisioned hardware MAC, preventing MAC spoofing attacks.
• Forged Transmits: Controls whether a VM can send outbound traffic with a source MAC address that differs from the provisioned virtual adapter MAC. Setting this to Reject prevents VMs from impersonating other nodes on the network.

---

VLAN Tagging Architectures at the Hypervisor Layer

Virtual switches enforce IEEE 802.1Q Virtual Local Area Network (VLAN) tagging to logically segment network traffic. There are three standard VLAN tagging models:

1. VIRTUAL SWITCH TAGGING (VST) - Standard Enterprise Model:
   [Guest VM] (Untagged) ---> [vSwitch Port Group: Tags VLAN 100] ---> [Physical Switch: 802.1Q Trunk]

2. EXTERNAL SWITCH TAGGING (EST) - Legacy Hardware Model:
   [Guest VM] (Untagged) ---> [vSwitch: VLAN 0 / Untagged] ---------> [Physical Switch: Access Port VLAN 100]

3. VIRTUAL GUEST TAGGING (VGT) - Virtual Appliance / Trunk Model:
   [Guest VM: 802.1Q Driver] (Tagged) -> [vSwitch: VLAN 4095/Trunk] -> [Physical Switch: 802.1Q Trunk]

1. Virtual Switch Tagging (VST)

• Implementation: The standard enterprise configuration. The physical switch port connecting to the server uplink is configured as an 802.1Q Trunk carrying multiple VLANs.
• Mechanism: Individual port groups on the virtual switch are assigned specific VLAN IDs (e.g., VLAN 10 for Web, VLAN 20 for App, VLAN 30 for DB). When a VM transmits an untagged Ethernet frame, the vSwitch port group intercepts the frame, appends the 802.1Q 4-byte VLAN tag, and transmits it out the physical uplink. In reverse, when tagged frames arrive from the physical switch, the vSwitch reads the tag, strips the 4-byte header, and delivers a standard untagged frame to the target VM.
• Advantage: Completely transparent to the guest OS; no special drivers or configurations are required inside the VM.

2. External Switch Tagging (EST)

• Implementation: All VLAN tagging is performed entirely by external physical network switches.
• Mechanism: The physical switch ports connecting to the server uplinks are configured as Access Ports assigned to a single specific VLAN. The hypervisor virtual switch is configured with VLAN ID 0 (or no VLAN tag). The vSwitch acts as a basic untagged software bridge.
• Limitation: Requires a dedicated physical network interface card (pNIC) and dedicated physical switch port for every single VLAN needed by the host. Highly inefficient; eliminates the cabling and consolidation benefits of virtualization.

3. Virtual Guest Tagging (VGT)

• Implementation: VLAN tagging is managed directly by the operating system running inside the virtual machine.
• Mechanism: The physical switch port is configured as an 802.1Q trunk, and the virtual switch port group is configured in trunk mode (assigned VLAN ID 4095 in VMware vSphere or configured for trunking in Hyper-V/KVM). The virtual switch passes all tagged 802.1Q frames directly through into the guest VM without altering the headers.
• Use Cases: Deployed exclusively for virtual network appliances: virtual firewalls (pfSense, Palo Alto VM-Series), virtual routers, multi-tenant load balancers, and packet capture probes that must process multiple VLAN tags natively on a single virtual network interface.

VLAN Tagging Models Comparison

---

Dedicated Hypervisor Network Infrastructure

To ensure security, deterministic latency, and fault tolerance, enterprise hypervisors mandate the logical or physical segmentation of hypervisor-level services into dedicated networks:

1. Management Network (VMkernel / Parent Partition Management): Carries administrative traffic, including web consoles, SSH, SNMP telemetry, API queries, and cluster heartbeat synchronization. Must be placed on an isolated management VLAN with strict access control lists (ACLs).
2. Live Migration Network (vMotion / Live Migration): Carries unencrypted (or encrypted), raw memory pages during live VM transitions. Generates massive burst traffic (saturating 10GbE or 25GbE links) and must be isolated on an unrouted, dedicated private network segment with Jumbo Frames (MTU 9000) enabled to minimize CPU packet processing overhead.
3. IP Storage Network (iSCSI / NFS): Transports block or file storage I/O between the hypervisor and SAN/NAS storage arrays. Requires isolated subnets, dedicated non-blocking network uplinks, multi-pathing (MPIO / Port Binding), and Jumbo Frames to prevent storage packet drops.
4. Fault Tolerance (FT) Logging Network: Carries real-time memory and CPU lockstep state replication between primary and secondary FT virtual machines. Requires extreme throughput and ultra-low latency (typically dual 10GbE links dedicated exclusively to FT logging).

---

Hardware Acceleration: Single Root I/O Virtualization (SR-IOV)

For ultra-high-throughput, latency-critical applications (such as telecommunications virtual network functions, algorithmic financial trading, and high-performance computing), the hypervisor software switch introduces unacceptable packet processing latency.

Single Root I/O Virtualization (SR-IOV) is a PCIe hardware specification that allows a single physical PCIe adapter to present itself as multiple independent physical devices directly to guest virtual machines:

• Physical Function (PF): The primary PCIe function of the physical network card. Supported by a full driver in the hypervisor host OS; used to configure, manage, and provision the adapter's virtual slices.
• Virtual Functions (VF): Lightweight, hardware-isolated PCIe functions carved out of the physical card. Each VF represents a virtual network interface with its own hardware transmit/receive queues, DMA channels, and PCIe configuration space.
• IOMMU Mapping (Direct Memory Access): Utilizing silicon virtualization extensions (Intel VT-d or AMD-Vi), the hypervisor assigns a VF directly to a guest VM's address space. The guest VM loads the vendor's physical NIC driver and communicates directly with the physical network adapter silicon via Direct Memory Access (DMA).

Standard Virtual Switching (High CPU, ~20-50 µs Latency):
[VM Application] -> [vNIC] -> [Hypervisor vSwitch Kernel] -> [Host Driver] -> [Physical NIC]

SR-IOV Direct Hardware Pass-Through (Near-Zero CPU, <2 µs Latency):
[VM Application] -> [VF Driver] ===(Direct DMA via IOMMU)===> [Physical NIC Hardware Queue]

[!CAUTION] The SR-IOV Operational Trade-off: By bypassing the hypervisor kernel, SR-IOV delivers line-rate 100GbE throughput and sub-microsecond latency with virtually zero host CPU utilization. However, because the VM is directly coupled to physical hardware registers, standard hypervisor services are disabled. The VM cannot utilize hypervisor-based distributed firewalls, vSwitch traffic shaping, virtual port mirroring, or standard live migration (vMotion) unless the hypervisor supports dynamic bonding of an SR-IOV VF to a paravirtualized fallback vNIC.

---

Virtual Storage Abstraction: vSAN, S2D, and Raw Device Mapping (RDM)

Virtual machines require resilient, highly available shared storage to support automated failover and dynamic resource scheduling.

Software-Defined Storage (Hyper-Converged Infrastructure - HCI)

Rather than connecting to expensive external Fibre Channel or iSCSI SAN arrays, modern enterprise architectures implement Software-Defined Storage (SDS), aggregating local storage across cluster nodes into a unified, resilient pool:

• VMware vSAN (Virtual SAN):
  • Embedded directly into the ESXi VMkernel. Pools local NVMe, SAS/SATA SSDs, and spinning HDDs across cluster hosts into a single shared distributed object datastore.
  • Tiering Architecture: Each host contributes to a Disk Group consisting of one high-end endurance flash drive for the Cache Tier (100% write buffer and read cache) and up to seven flash or magnetic drives for the Capacity Tier.
  • Storage Policy Based Management (SPBM): Resiliency is configured per-VM using declarative policies: Failures to Tolerate (FTT=1, FTT=2) implemented via RAID-1 mirroring or RAID-5/6 distributed erasure coding. Requires a dedicated low-latency 10GbE/25GbE network and an external Witness Node in 2-node or stretched clusters to prevent split-brain scenarios.
• Microsoft Storage Spaces Direct (S2D):
  • The software-defined storage technology integrated into Windows Server Datacenter.
  • Aggregates local host drives across Windows Server Failover Cluster nodes using the Software Storage Bus (SSB).
  • Exposes resilient Cluster Shared Volumes (CSV) using the high-performance SMB3 / SMB Direct protocol over Remote Direct Memory Access (RDMA, via RoCE or iWARP) network adapters, providing line-rate storage throughput with minimal CPU overhead.

Raw Device Mapping (RDM)

A Raw Device Mapping (RDM) is a specialized pointer file (.vmdk) stored on a hypervisor datastore that acts as a symbolic proxy, mapping a physical Storage Area Network (SAN) LUN directly to a virtual machine, bypassing the VMFS filesystem.

RDMs operate in two compatibility modes:

1. Virtual Compatibility Mode (vRDM): The hypervisor intercepts all SCSI commands sent by the guest VM. It allows SCSI read and write commands to pass directly to the SAN LUN while virtualizing commands related to inquiry and metadata. Advantage: Supports hypervisor-level snapshotting, cloning, and standard vMotion.
2. Physical Compatibility Mode / Pass-Through RDM (pRDM): The hypervisor completely steps aside. All SCSI commands (except basic LUN inquiry) pass directly through to the physical SAN storage controller without modification.
   • Primary Use Cases: Required for physical-to-virtual failover clusters running quorum disks (e.g., WSFC spanning a physical server and a VM), storage management software running inside a VM that sends proprietary SCSI commands to the array, and SAN-aware hardware Volume Shadow Copy Service (VSS) snapshot integration.
   • Constraint: Does not support hypervisor-level VM snapshots.

---

Virtual Machine Migration: Live Migration & Storage Migration

Virtual machine mobility enables seamless hardware maintenance, dynamic resource rebalancing, and automated disaster recovery without impacting running services.

Live Migration Mechanics (VMware vMotion / Hyper-V Live Migration)

Live Migration moves an active, powered-on virtual machine from one physical hypervisor host to another across a local network with zero perceived downtime and no dropped TCP connections.

[SOURCE HOST]                                                   [TARGET HOST]
[Active VM]                                                     [Pre-Allocated VM]
   |                                                                   |
   |--- Phase 1: Pre-Copy (Iterative RAM Sync over vMotion Net) ------->|
   |                                                                   |
   |--- Phase 2: Memory Bitmap Tracking (Captures Dirty Pages) -------->|
   |                                                                   |
   |--- Phase 3: Stun & Final Cutover (<1 sec, transfers CPU state) --->|
   |                                                                   |
   X (VM De-registered)                             [VM Resumes & Broadcasts GARP]
                                                                       |
                                                    vSwitch Updates MAC Table

The Five-Phase Live Migration Process

1. Phase 1: Pre-Migration Compatibility Check: The cluster management engine validates that the target host possesses sufficient unreserved CPU and RAM, access to the identical shared datastore, identical virtual switch port group names and VLAN IDs, and an instruction-compatible CPU architecture.
2. Phase 2: Initial Memory Pre-Copy: A memory state copy begins. While the virtual machine continues executing normally on the source host, the hypervisor copies the entire physical RAM footprint of the VM across the dedicated vMotion network to the target host.
3. Phase 3: Iterative Dirty Page Synchronization: Because the VM continues processing active transactions during Phase 2, it modifies memory pages. The hypervisor marks these altered pages as "dirty" in an internal memory bitmap. The hypervisor transmits the dirty pages to the target host in iterative rounds. Each round is smaller and faster than the preceding round.
4. Phase 4: VM Stun and Final State Cutover: When the rate of remaining dirty pages drops below a threshold that can be transferred in a fraction of a second, the hypervisor initiates the cutover. The VM on the source host is briefly stunned (quiesced) for less than 500 milliseconds. During this stun window, the final memory delta and exact CPU register execution states (instruction pointer, registers, stack pointers) are transferred to the target host.
5. Phase 5: Network Rerouting and Resume: The VM resumes execution on the target host. The target host's virtual switch immediately transmits a Gratuitous Address Resolution Protocol (GARP) or Reverse ARP (RARP) broadcast onto the physical network fabric. Top-of-Rack physical switches receive the GARP, update their internal MAC address forwarding tables (CAM tables) to point the VM's MAC address to the target host's physical switch port, and traffic flows to the new host without interrupting active TCP sessions.

Live Migration Prerequisites

Successful live migration requires strict environmental prerequisites:

• Shared Clustered Storage: Virtual disk files must reside on shared storage (SAN via FC/iSCSI, NAS via NFS, or HCI via vSAN/S2D) accessible simultaneously with identical LUN/datastore permissions by both source and destination hosts.
• Dedicated High-Speed Network: Minimum 1 Gbps Ethernet connection, with 10 Gbps or 25 Gbps strongly recommended, operating with round-trip latency under 5 milliseconds (or up to 150 ms for long-distance/cross-vCenter vMotion).
• Network Port Group Parity: Virtual switch port groups on source and target hosts must have identical names (case-sensitive) and access to the identical Layer 2 broadcast domains (same VLAN IDs).
• Processor Architecture Compatibility (Enhanced vMotion Compatibility - EVC): Both hosts must share compatible processor families (Intel to Intel, AMD to AMD).

Enhanced vMotion Compatibility (EVC)

When a cluster contains physical servers purchased across different hardware refresh cycles, their processors contain different instruction set extensions (e.g., Host A has Intel Broadwell supporting AVX2; Host B has Intel Ice Lake supporting AVX-512).

• The Problem: If a virtual machine boots on an Ice Lake host, the guest operating system detects and utilizes AVX-512 instructions. If the VM is live-migrated to a Broadwell host that lacks AVX-512 execution pipelines, the next time the guest OS attempts to execute an AVX-512 instruction, the CPU generates an invalid opcode exception, causing an immediate guest OS kernel panic or Blue Screen of Death.
• The Solution: Enhanced vMotion Compatibility (EVC) masks out newer instruction set extensions at the hypervisor level. When an EVC baseline (e.g., Intel "Broadwell Generation") is applied to a cluster, all hosts—regardless of how new their physical CPUs are—expose only instructions supported up to that baseline to guest VMs. This guarantees complete CPU instruction uniformity across heterogeneous server hardware, enabling seamless live migration.

Storage Live Migration (Storage vMotion)

While Live Migration transfers CPU and memory states across hosts while keeping storage pinned to a shared datastore, Storage Live Migration (VMware Storage vMotion) moves a running virtual machine's virtual disk files from one datastore to another without interrupting VM execution:

• Mechanism: Uses a real-time I/O Mirroring Driver. The target disk is created on the new datastore, and historical data blocks are sequentially copied from the source to the destination. Concurrently, all new incoming write requests from the running VM are mirrored simultaneously to both the source and target disks. Once the background synchronization is complete, the hypervisor switches disk access cleanly to the target datastore and removes the source files.

---

Migration Workflows: P2V and V2V Conversions

Transitioning legacy workloads into modern virtual infrastructure requires structured conversion methodologies:

Physical-to-Virtual (P2V) Conversions

P2V conversion decouples an operating system and application stack installed on a physical bare-metal server, packaging it into a virtual machine image (VMDK, VHDX, or QCOW2).

1. Hot (Online) P2V Conversion:
   • An agent utility (e.g., VMware vCenter Converter Standalone) is installed inside the active running physical server OS.
   • Utilizes the operating system's snapshot mechanism (Microsoft Volume Shadow Copy Service / VSS in Windows, or LVM snapshots in Linux) to capture a point-in-time image of system and data volumes while applications remain running.
   • Trade-off: Minimal downtime, but transactional workloads (such as Microsoft Exchange or transactional databases) can suffer data inconsistencies if writes occur between snapshot generation and final cutover.
2. Cold (Offline) P2V Conversion:
   • The physical server is shut down and booted from an independent, live conversion media (such as a WinPE or Linux bootable ISO image).
   • The conversion software clones physical disk blocks sector-by-sector directly to a virtual disk on the hypervisor datastore.
   • Advantage: Absolute quiescence. Zero data drift, zero risk of open file locking errors, and guaranteed database transaction consistency.

Virtual-to-Virtual (V2V) Conversions

V2V conversion transforms virtual machines from one hypervisor format to another (e.g., migrating from VMware ESXi to Microsoft Hyper-V or KVM).

• Driver Injection and Scrubbing: The primary challenge in V2V conversions is driver incompatibility. A VM running on ESXi possesses SCSI controller drivers tailored for LSI Logic SAS or VMware Paravirtual SCSI (pvscsi). If converted to KVM or Hyper-V without modification, the new hypervisor cannot initialize the virtual storage controller, resulting in an immediate stop error: 0x0000007B: INACCESSIBLE_BOOT_DEVICE.
• Execution Protocol: Conversion tools (e.g., qemu-img, StarWind V2V Converter) inject the destination hypervisor's paravirtualized storage and network drivers (e.g., VirtIO drivers or Hyper-V Integration Services) into the guest operating system's driver repository, adjust bootloader configurations, and gracefully uninstall the source hypervisor's guest tools.

Guest Connectivity Modes: Direct Access (Bridged), NAT, Host-Only, and Internal

Before VLAN tagging or SR-IOV enters the picture, every virtual machine's vNIC is attached in one of four connectivity modes. SK0-005 names the first two explicitly, and scenario questions are usually resolved by identifying which mode is in use.

Bridged (direct access) is the mode production server virtualization uses almost exclusively. The virtual switch forwards the guest's frames onto the physical uplink with the guest's own MAC address, so the VM appears on the wire as an independent machine: it receives an address from the same DHCP scope as physical servers, is reachable by hostname through normal DNS, and can host inbound services without any translation. The operational consequences follow directly — the guest consumes a real IP from the production subnet, its MAC must be permitted by any switch port security configuration, and the physical switch port must be a trunk if the guest carries tagged VLAN traffic.

NAT places the guest behind the hypervisor, which performs address translation exactly as a home router does. Outbound sessions work transparently, but nothing on the LAN can initiate a connection to the VM unless a port-forwarding rule is created. This is why NAT is standard on workstation-class hypervisors and effectively never used for production server workloads: an application server nobody can reach is not a server. The classic exam scenario describes a newly deployed VM with working internet access whose service is unreachable from other machines — that combination is the signature of NAT where bridged was required.

Host-only and internal/private networks carry no path to the physical LAN at all. Their production relevance is the isolated network bubble used for simulated failover testing, where recovery VMs are powered on with identical IP addresses to production without any possibility of an address conflict or of test traffic reaching real users.