2.1 Virtualization & Hypervisor Architectures

Virtualization & Hypervisor Architectures

Virtualization is the foundational technology that enables cloud computing. By abstracting physical compute, memory, storage, and networking hardware into software-defined resource pools, cloud service providers (CSPs) and enterprise private clouds can dynamically provision, scale, and isolate multi-tenant workloads with unprecedented density and efficiency.

For the CompTIA Cloud+ (CV0-004) exam, cloud engineers must master the internal mechanics of hypervisor architectures, hardware-assisted virtualization extensions, memory management techniques, high-performance I/O virtualization, and virtual machine (VM) lifecycle management.

---

1. Hypervisor Architecture: Type 1 (Bare-Metal) vs. Type 2 (Hosted)

A hypervisor (or Virtual Machine Monitor - VMM) is the software, firmware, or hardware layer responsible for creating, executing, and managing virtual machines. The hypervisor presents guest operating systems with a virtualized hardware environment and arbitrates access to the underlying physical CPU, memory, storage, and network interfaces.

Hypervisors fall into two primary architectural classifications:

+-----------------------------------------------------------------------------+
|                   TYPE 1 VS. TYPE 2 HYPERVISOR ARCHITECTURE                 |
|                                                                             |
|       TYPE 1 (BARE-METAL / NATIVE)                TYPE 2 (HOSTED)           |
|   +---------------------------------+   +---------------------------------+ |
|   |   VM 1   |   VM 2   |   VM 3    |   |   VM 1   |   VM 2   |   VM 3    | |
|   | (App/OS) | (App/OS) | (App/OS)  |   | (App/OS) | (App/OS) | (App/OS)  | |
|   +---------------------------------+   +---------------------------------+ |
|   |    TYPE 1 HYPERVISOR (VMM)      |   |    TYPE 2 HYPERVISOR (VMM)      | |
|   |   (VMkernel / Hyper-V / KVM)    |   |  (VirtualBox / Workstation)     | |
|   +---------------------------------+   +---------------------------------+ |
|   |        PHYSICAL HARDWARE        |   |       HOST OPERATING SYSTEM     | |
|   |      (CPU, RAM, NIC, Disk)      |   |    (Windows / Linux / macOS)    | |
|   +---------------------------------+   +---------------------------------+ |
|                                         |        PHYSICAL HARDWARE        | |
|                                         |      (CPU, RAM, NIC, Disk)      | |
|                                         +---------------------------------+ |
+-----------------------------------------------------------------------------+

Type 1: Bare-Metal (Native) Hypervisors

Type 1 hypervisors install directly onto the physical bare-metal server hardware without an intermediate general-purpose operating system. The hypervisor kernel has direct access to CPU execution rings, memory controllers, and PCIe buses.

• Enterprise Implementations:
  • VMware ESXi: Proprietary monolithic hypervisor running the specialized VMkernel microkernel.
  • Microsoft Hyper-V: Microkernel hypervisor where the hypervisor sits directly on hardware, and the primary management partition (Parent Partition running Windows Server) controls guest VM child partitions.
  • KVM (Kernel-based Virtual Machine): Integrates directly into the mainline Linux kernel via loadable kernel modules (kvm.ko and kvm-intel.ko / kvm-amd.ko), effectively turning the Linux kernel itself into a Type 1 bare-metal hypervisor.
  • Xen / AWS Nitro Hypervisor: Historically powered AWS EC2 (Xen PV/HVM); modern AWS infrastructure utilizes the AWS Nitro System, offloading virtualization functions (networking, EBS storage, security, management) to dedicated hardware PCIe cards, leaving almost 100% of host CPU and memory for customer VMs.
• Operational Characteristics: Maximum performance, minimal virtualization overhead (typically 1–3%), enterprise clustering, live migration (VMware vMotion, Hyper-V Live Migration, KVM Live Migration), high availability (HA), and multi-tenant security isolation.

Type 2: Hosted Hypervisors

Type 2 hypervisors execute as user-space applications on top of an existing host operating system (such as Windows 11, macOS, or Ubuntu Linux). When a guest VM requests CPU cycles, memory allocations, or network packets, the request must traverse the guest OS, the Type 2 hypervisor, the host OS kernel, and finally the physical hardware.

• Common Implementations: Oracle VM VirtualBox, VMware Workstation Pro / Fusion, Parallels Desktop, QEMU (when run in user-space emulation mode).
• Operational Characteristics: Increased CPU overhead (10–25%), double-scheduling latency (hypervisor scheduler competing with host OS scheduler), dependency on host OS stability and patch reboots. Ideal for local software development, lab testing, security sandboxing, and desktop multi-boot scenarios, but strictly unsuitable for production enterprise cloud infrastructure.

Architectural Comparison Matrix

---

2. Hardware-Assisted Virtualization & CPU Scheduling

Early x86 virtualization required complex software techniques such as binary translation and paravirtualization because certain x86 CPU instructions (Ring 0 privileged instructions) could not be safely virtualized without trapping. Modern enterprise virtualization relies on silicon-level hardware extensions built into processor architectures.

CPU Hardware Extensions: Intel VT-x and AMD-V

Hardware-assisted CPU virtualization introduces a new hardware execution privilege level known as VMX Root and VMX Non-Root mode:

• Intel VT-x (Virtualization Technology for x86): Adds the VMCS (Virtual Machine Control Structure), which automatically handles context switching between the hypervisor (VMX root) and guest operating systems (VMX non-root) through hardware VMENTRY and VMEXIT instructions.
• AMD-V (AMD Virtualization): Uses the VMCB (Virtual Machine Control Block) to manage state switching and guest execution.

Second Level Address Translation (SLAT)

In physical computing, the CPU Memory Management Unit (MMU) translates virtual memory addresses used by applications into physical RAM addresses. In a virtualized environment, a double translation is required: Guest Virtual Address (GVA) $\rightarrow$ Guest Physical Address (GPA) $\rightarrow$ Host Physical Address (HPA).

SLAT implements hardware-level page tables in the CPU silicon to handle this two-stage translation without hypervisor software intervention:

• Intel EPT (Extended Page Tables): Hardware MMU traverses guest and host page tables simultaneously, drastically reducing the performance penalty of memory-intensive workloads.
• AMD RVI / NPT (Rapid Virtualization Indexing / Nested Page Tables): AMD's hardware implementation of SLAT.

vCPU Overcommitment, Allocation Ratios & CPU Contention

A virtual CPU (vCPU) represents a virtualized execution thread presented to the guest OS. Hypervisors schedule vCPUs as software execution threads on physical CPU cores (pCores) or hyperthreaded logical processors (pThreads).

vCPU Overcommitment allows administrators to assign more aggregate vCPUs across running VMs than the physical host possesses (e.g., allocating 64 vCPUs across 16 VMs on a physical host with only 16 logical cores, yielding a 4:1 overcommitment ratio).

+-----------------------------------------------------------------------------+
|                     vCPU SCHEDULING & CONTENTION METRICS                    |
|                                                                             |
|   PHYSICAL CPU CORES: [Core 0]  [Core 1]  [Core 2]  [Core 3]                |
|                             ^         ^         ^         ^                 |
|                             |         |         |         | (Hypervisor     |
|                             +----+----+----+----+         |  Scheduler)     |
|                                  |         |              |                 |
|   RUNNING vCPUs:               [vCPU 1]  [vCPU 2]      [vCPU 3]  [vCPU 4]   |
|   WAITING vCPUs (Contention):  [vCPU 5]  [vCPU 6]  <-- Stalled waiting!     |
|                                                                             |
|   - VMware Metric: CPU Ready (%RDY) > 5% per vCPU = CPU Starvation          |
|   - Linux Metric:  CPU Steal (%st) in 'top' > 3-5% = Host Contention        |
+-----------------------------------------------------------------------------+

Key Performance Contention Metrics:

1. CPU Ready Time (%RDY in VMware ESXi / esxtop): The percentage of time a guest VM had a vCPU ready to execute instructions but had to wait in the hypervisor scheduling queue because all physical cores were occupied by other workloads. In enterprise environments, a %RDY exceeding 5% to 10% per vCPU indicates severe CPU overcommitment and degrades application performance.
2. CPU Steal Time (%st in Linux top / sar / mpstat): The percentage of time a virtualized Linux instance wanted to execute CPU instructions but physical CPU cycles were "stolen" by the hypervisor to serve other tenants. In cloud instances (such as AWS EC2 or Azure VMs), high CPU steal indicates a noisy neighbor or an oversubscribed hypervisor host.
3. Co-Stop (%CSTP): Occurs when a multi-vCPU VM experiences skew because one vCPU executes faster than sibling vCPUs, forcing the hypervisor to pause the faster vCPU to allow lagging vCPUs to catch up.

[!IMPORTANT] Cloud+ Sizing Principle (Right-Sizing vCPUs): Allocating more vCPUs to a virtual machine than required does not improve performance and frequently degrades it. A 8-vCPU VM on an overcommitted host must wait for the hypervisor scheduler to find 8 available pCores simultaneously (relaxed co-scheduling), causing excessive %RDY and %CSTP wait times. Always start with minimal vCPUs and scale based on monitored utilization.

Non-Uniform Memory Access (NUMA) & CPU Pinning

Modern enterprise servers feature multi-socket processor architectures where each CPU socket has directly attached, high-speed local memory controllers. This architecture is called NUMA (Non-Uniform Memory Access).

• NUMA Node: A physical socket paired with its dedicated local memory bank.
• Local Access vs. Remote Access: A core accessing RAM attached to its own socket experiences low latency (~50-80 ns). Accessing RAM attached to a different socket via processor interconnects (Intel UPI / AMD Infinity Fabric) introduces significant memory access latency penalties (150-250+ ns).
• vNUMA (Virtual NUMA): Exposes the physical NUMA topology directly into the guest OS for wide VMs (VMs larger than a single physical NUMA node), allowing the guest OS memory manager to optimize thread placement.
• CPU Pinning / vCPU Affinity: Explicitly binding specific vCPUs to dedicated physical processor cores and memory nodes. This eliminates context-switching latency and guarantees zero cross-socket NUMA traversals, critical for ultra-low latency financial trading, high-frequency packet processing, and telecommunications NFV (Network Functions Virtualization).

# Linux KVM CPU Pinning with virsh
# Pin vCPU 0 of VM 'db-prod-01' to physical core 4, and vCPU 1 to physical core 5
virsh vcpupin db-prod-01 0 4 --config --live
virsh vcpupin db-prod-01 1 5 --config --live

# Verify vCPU affinity and NUMA alignment
virsh vcpuinfo db-prod-01

---

Host Affinity & Anti-Affinity Rules (VM Placement Policies)

Enterprise hypervisor clusters (VMware vSphere DRS, Hyper-V Failover Clustering, OpenStack Nova schedulers) and cloud placement groups control which physical hosts run which virtual machines through affinity policies:

• Host Affinity (VM-to-Host Pinning): Rules that constrain a VM to run only on a specific host or host group. Common drivers include per-socket software licensing (pinning Oracle Database VMs to two licensed hosts prevents the license scope from expanding across the entire cluster), NUMA locality compliance, and regulatory requirements tying workloads to specific hardware.
• VM-to-VM Affinity ("keep together"): Co-locates tightly coupled VMs — such as an application server and its database — on the same physical host, eliminating virtual-switch and physical-network hops and minimizing inter-VM communication latency.
• VM-to-VM Anti-Affinity ("keep apart"): Forces redundant VMs onto different physical hosts so a single motherboard, power supply, or rack failure cannot destroy both members of a highly available pair simultaneously. Cloud equivalents include AWS Spread Placement Groups (each instance on distinct rack hardware) and Azure Availability Sets distributed across Fault Domains and Update Domains.
• Trade-off: Aggressive host-affinity constraints reduce cluster flexibility — a pinned VM cannot be evacuated by live migration (vMotion) during host maintenance unless the rule is first suspended, and over-constrained placement can prevent HA restarts when capacity is scarce.

[!IMPORTANT] Exam Trap: Anti-affinity rules protect against correlated hardware failure (two replicas dying on the same host); they do not improve software performance. Affinity rules serve licensing, latency, and data-locality goals.

---

3. Memory Virtualization, Overcommitment & Reclamation

Hypervisors manage physical RAM and allocate virtual memory spaces to guest VMs. To optimize density and reduce hardware costs, hypervisors employ advanced memory overcommitment and reclamation algorithms.

+-----------------------------------------------------------------------------+
|                     MEMORY RECLAMATION HIERARCHY                            |
|                                                                             |
|   [HYPERVISOR DETECTS MEMORY PRESSURE ON HOST]                              |
|         |                                                                   |
|         v                                                                   |
|   1. TRANSPARENT PAGE SHARING (TPS / KSM)  ---> Zero guest performance loss |
|      (Deduplicates identical 4KB OS memory pages across VMs via CoW)        |
|         |                                                                   |
|         +---> If host memory pressure persists...                           |
|         v                                                                   |
|   2. MEMORY BALLOONING                     ---> Minimal / Controlled impact |
|      (Guest balloon driver inflates, forcing guest OS to page its own cache)|
|         |                                                                   |
|         +---> If host memory pressure persists...                           |
|         v                                                                   |
|   3. MEMORY COMPRESSION                    ---> Moderate CPU overhead       |
|      (Compresses evicted pages into a local host RAM cache)                 |
|         |                                                                   |
|         +---> If host memory is completely exhausted...                     |
|         v                                                                   |
|   4. HYPERVISOR SWAPPING                   ---> CATASTROPHIC LATENCY SPIKE  |
|      (Hypervisor writes raw guest memory to host disk .vswp file)           |
+-----------------------------------------------------------------------------+

Memory Management Mechanisms

1. Memory Ballooning:
   • A pseudo-device driver (part of VMware Tools, Hyper-V Integration Services, or VirtIO guest drivers) is installed inside the guest OS.
   • When the host runs low on physical memory, the hypervisor instructs the balloon driver to "inflate" by allocating memory from the guest OS.
   • The guest OS kernel uses its standard memory management routines to free clean disk cache pages or page out idle user-space memory to the guest OS swap file.
   • The balloon driver captures these freed physical host pages and returns them to the hypervisor pool.
   • When pressure subsides, the balloon driver "deflates", returning memory to the guest.
2. Transparent Page Sharing (TPS) / Kernel Samepage Merging (KSM):
   • In multi-tenant environments running multiple identical guest operating systems (e.g., 50 instances of Ubuntu 24.04), hundreds of megabytes of read-only OS binaries and libraries in RAM are identical.
   • The hypervisor scans guest memory pages, hashes their contents, and detects identical 4KB pages.
   • It consolidates duplicate pages into a single physical memory frame marked as read-only and maps all guest VMs to this shared page.
   • When any guest attempts to write to the page, a Copy-on-Write (CoW) fault is triggered, and the hypervisor transparently creates a private, writable copy for that VM.
3. Hypervisor Swapping:
   • If memory ballooning and TPS fail to alleviate host memory starvation, the hypervisor is forced to write guest physical memory directly to a hypervisor swap file on disk (e.g., .vswp in VMware ESXi).
   • Exam Warning: Hypervisor swapping bypasses the guest OS kernel completely. The hypervisor does not know which guest pages are active or idle, frequently swapping active kernel data structures to disk, which induces massive I/O thrashing and multi-second system freezes.

---

4. I/O Virtualization, vSwitching & SR-IOV

Network and storage I/O performance in virtualized environments is governed by the virtualization path between guest device drivers and physical hardware adapters.

+-----------------------------------------------------------------------------+
|              SOFTWARE vSWITCH VS. HARDWARE SR-IOV NETWORKING                |
|                                                                             |
|   STANDARD SOFTWARE vSWITCH                 HARDWARE SR-IOV (DIRECT DMA)    |
|   +--------------------------+              +--------------------------+    |
|   |    VM GUEST OS (vNIC)    |              |    VM GUEST OS (VF Driver) |   |
|   +--------------------------+              +--------------------------+    |
|                | (virtio/e1000)                          |                  |
|                v                                         | (Direct PCIe DMA |
|   +--------------------------+                           |  Bypasses        |
|   | HYPERVISOR vSWITCH       |                           |  Hypervisor!)    |
|   | - Packet copying in RAM  |                           |                  |
|   | - Software VLAN tagging  |                           |                  |
|   | - CPU overhead & context |                           |                  |
|   +--------------------------+                           |                  |
|                |                                         |                  |
|                v                                         v                  |
|   +--------------------------+              +--------------------------+    |
|   | PHYSICAL NIC (pNIC)      |              | PHYSICAL NIC (PF)        |    |
|   | - Shared uplink port     |              | - Carved into VFs (VF0..)|    |
|   +--------------------------+              +--------------------------+    |
+-----------------------------------------------------------------------------+

Virtual Switching (vSwitch)

A virtual switch (vSwitch) is a software abstraction running inside the hypervisor that logically connects virtual network interfaces (vNICs) to each other and to physical network adapters (pNICs / uplinks).

• Standard Virtual Switch (vSwitch): Configured and managed locally on an individual hypervisor host.
• Distributed Virtual Switch (vDS): Extends switch configuration (VLANs, traffic shaping, security policies, private VLANs) across an entire cluster of hypervisor hosts, ensuring consistent network configuration during VM live migration.
• VLAN Tagging Methods:
  • Virtual Switch Tagging (VST): The vSwitch strips and adds 802.1Q VLAN tags as packets ingress/egress the physical uplink (most common enterprise design).
  • Virtual Guest Tagging (VGT): The guest OS handles 802.1Q tagging directly; the vSwitch passes trunked VLAN traffic straight into the VM vNIC.
  • External Switch Tagging (EST): Physical top-of-rack (ToR) switches handle VLAN tagging; all packets reach the hypervisor untagged.

Single Root I/O Virtualization (SR-IOV)

Standard virtual switching requires the hypervisor CPU to inspect every packet header, copy payloads across memory buffers, and manage software queues. For high-throughput (25G/40G/100Gbps) and latency-sensitive workloads (such as Cassandra, Kafka, Redis, or VoIP gateways), software switching introduces latency and consumes substantial CPU cycles.

SR-IOV is a PCIe hardware specification that allows a single physical PCIe device to appear as multiple separate physical devices:

• Physical Function (PF): The primary PCIe function supporting full device management, SR-IOV configuration, and host OS control.
• Virtual Function (VF): A lightweight PCIe function containing dedicated registers and queues. The hypervisor assigns a VF directly to a guest VM using PCI Passthrough / Direct Memory Access (DMA).
• Benefits: Line-rate packet processing, sub-microsecond latency, near-zero hypervisor CPU overhead.
• Trade-offs: Bypasses the software vSwitch entirely (disabling vSwitch-based packet inspection, security filtering, and netflow metrics) and complicates live migration unless bonded with a paravirtualized failover vNIC.
• Cloud Implementations: AWS Enhanced Networking (Elastic Network Adapter - ENA), Azure Accelerated Networking, GCP gVNIC.

---

5. Virtual Machine Anatomy, Disks & Storage Formats

A virtual machine is encapsulated as a set of discrete configuration, state, and disk image files stored on enterprise shared storage (SAN, NAS, or Cloud Block Storage).

Virtual Disk Image Formats

Provisioning Types: Thin vs. Thick Provisioning

• Thin Provisioning: Allocates storage capacity on demand as data is written by the guest OS. Enables storage oversubscription, but risks volume read-only freeze if the underlying storage datastore runs out of physical capacity.
• Thick Provisioning Lazy-Zeroed: Allocates all required storage upfront on the datastore. Space is reserved, but disk blocks are only wiped (zeroed out) when the guest OS writes to them for the first time.
• Thick Provisioning Eager-Zeroed: Allocates all storage upfront and completely wipes all blocks with zeroes during disk creation. Provides highest I/O performance and eliminates write-time zeroing latency (mandatory for Microsoft Failover Clustering and VMware FT).

Snapshots vs. Backups: Critical Architectural Distinction

[!CAUTION] Exam Trap — Snapshots Are NEVER Backups: One of the most tested concepts on the CompTIA Cloud+ exam is the fundamental distinction between a snapshot and a backup. A snapshot is a point-in-time state capture relying on differencing (delta) disks. When a snapshot is taken:

1. The original base virtual disk (base.vmdk) is locked into a read-only state.
2. A new delta differencing file (base-000001-delta.vmdk) is created to record all subsequent new writes.
3. Read requests must search the delta file first; if the block is not found, the hypervisor reads from the base disk.

+-----------------------------------------------------------------------------+
|                 SNAPSHOT DIFFERENCING DISK DELTA MECHANICS                  |
|                                                                             |
|   [BASE VIRTUAL DISK]  <--- LOCKED AS READ-ONLY                             |
|         ^                                                                   |
|         | (Read fallback if block not modified)                             |
|   [SNAPSHOT 1 DELTA]   <--- Read-Only Delta (Snapshot Taken at 10:00 AM)    |
|         ^                                                                   |
|         | (Read fallback if block not modified)                             |
|   [SNAPSHOT 2 DELTA]   <--- Read-Only Delta (Snapshot Taken at 12:00 PM)    |
|         ^                                                                   |
|         |                                                                   |
|   [ACTIVE DELTA DISK]  <--- ALL CURRENT WRITES GO HERE                      |
+-----------------------------------------------------------------------------+

Why Snapshots Must Not Be Used for Long-Term Data Protection:

1. Dependent Integrity: If the base virtual disk is corrupted or deleted, all downstream snapshots are permanently lost.
2. Storage Exhaustion: Delta files grow dynamically with every modified block and can rapidly consume all available datastore capacity.
3. I/O Latency Degradation: Long snapshot trees force the hypervisor to traverse multiple delta chains for a single read operation, severely increasing disk latency.
4. Commit Penalty: Deleting/merging large snapshots requires heavy read/write I/O to consolidate delta blocks into the base disk, risking performance degradation on production databases.
5. True Backup Definition: A backup is a standalone, independent copy of VM data, configuration, and state stored on a separate physical storage system or remote cloud repository, adhering to the 3-2-1 backup rule.

---

6. Bare-Metal Cloud Instances vs. Virtualized Compute

All major cloud providers (AWS, Microsoft Azure, Google Cloud Platform, IBM Cloud) offer Bare-Metal Cloud Instances alongside standard virtualized multi-tenant instances.

When to Architect for Bare-Metal Cloud Compute:

1. Elimination of Hypervisor Overhead & "Noisy Neighbor" Jitter: Eliminates all CPU scheduling queues, %RDY times, and memory overcommitment penalties, providing 100% dedicated hardware throughput for ultra-high-performance computing (HPC), machine learning training, and high-frequency transaction processing.
2. Nested Virtualization Requirements: Running custom Type 1 hypervisors (e.g., VMware Cloud on AWS, Nutanix on Azure) or specialized hypervisor configurations (custom KVM/Xen builds) that cannot run efficiently inside standard guest VMs.
3. Hardware Counter & Non-Virtualizable Hardware Access: Applications requiring direct access to CPU Performance Monitoring Units (PMUs), specialized hardware security modules (HSMs), or proprietary PCIe accelerators.
4. Restrictive Software Licensing Models: Enterprise software (such as Oracle Database, SAP HANA, or Microsoft SQL Server) whose software licensing models mandate per-physical-socket or per-physical-core licensing rather than vCPU licensing.

---

7. CompTIA Cloud+ Exam Tips & Troubleshooting Scenarios

• High CPU Steal in Guest VM: If a cloud Linux VM shows high %st in top while application workload is moderate, the physical hypervisor host is overcommitted. Remedy: Scale horizontally, migrate the VM to another host, or upgrade to a dedicated compute instance family.
• Storage Datastore Full Due to Snapshots: If a datastore runs out of disk space, all VMs on that datastore will freeze in a read-only state. Check for orphaned VM snapshots that were never merged after automated backup routines.
• NUMA Inefficiency Warning: When provisioning a large VM (e.g., 32 vCPUs on a server with two 16-core sockets), ensure virtual NUMA (vNUMA) is enabled. If vNUMA is disabled, the guest OS will treat the entire memory space as Uniform, leading to high cross-socket remote memory access penalties.