8.2 NVIDIA vGPU Software & Virtualization Models

8.2 NVIDIA vGPU Software & Virtualization Models

Executive Summary: Enterprise IT organizations frequently standardize on virtualized environments—such as VMware vSphere/ESXi, Red Hat OpenShift Virtualization (KVM), and Nutanix AHV—to achieve centralized management, workload migration (vMotion), snapshotting, and multi-tenant security. NVIDIA Virtual GPU (vGPU) software extends native, hardware-accelerated GPU capabilities directly into virtual machines (VMs), allowing physical data center GPUs to be dynamically partitioned and shared across hundreds of concurrent virtual workloads.

---

1. NVIDIA vGPU Architecture & Software Stack

The NVIDIA vGPU software architecture bridges the physical hardware layer and virtualized guest operating systems through a two-tier software design.

                      NVIDIA vGPU SOFTWARE STACK ARCHITECTURE

  ┌──────────────────────────────────┐      ┌──────────────────────────────────┐
  │     GUEST VM 1 (Linux / RHEL)    │      │    GUEST VM 2 (Windows Server)   │
  │  ┌────────────────────────────┐  │      │  ┌────────────────────────────┐  │
  │  │ User App (PyTorch / CUDA)  │  │      │  │ CAD / Visual App (DirectX) │  │
  │  ├────────────────────────────┤  │      │  ├────────────────────────────┤  │
  │  │ NVIDIA vGPU Guest Driver   │  │      │  │ NVIDIA vGPU Guest Driver   │  │
  │  │ (CUDA, NVENC, Vulkan, NLS) │  │      │  │ (DirectX, OpenGL, NLS)     │  │
  │  └─────────────┬──────────────┘  │      │  └─────────────┬──────────────┘  │
  └────────────────┼─────────────────┘      └────────────────┼─────────────────┘
                   │ Virtual PCI Pass-through (VFIO / mdev)  │
  ═════════════════╪═════════════════════════════════════════╪══════════════════
  HYPERVISOR LAYER │ (VMware ESXi / Red Hat KVM / Citrix)    │
  ┌────────────────┴─────────────────────────────────────────┴─────────────────┐
  │              NVIDIA vGPU MANAGER (Host Kernel Module / VIB)               │
  │    - Mediated Device (mdev) Lifecycle Management                           │
  │    - MMIO Interception & Virtual Configuration Space Emulation             │
  │    - Time-Slice Scheduler / Hardware MIG Profile Mapper                    │
  └─────────────────────────────────────┬──────────────────────────────────────┘
                                        │ Direct Hardware Access
  ══════════════════════════════════════╪══════════════════════════════════════
  PHYSICAL HARDWARE LAYER               │
  ┌─────────────────────────────────────┴──────────────────────────────────────┐
  │        Physical Data Center GPU (NVIDIA A100 / H100 / L40S / A40)          │
  └────────────────────────────────────────────────────────────────────────────┘

1. Host Component: NVIDIA vGPU Manager

• Role: Installed directly into the hypervisor operating system (e.g., as a .vib package on VMware ESXi or an RPM module on Red Hat KVM).
• Functionality: The vGPU Manager acts as the hardware mediator. It interfaces with the Linux kernel Mediated Device framework (mdev) or VMware DirectPath I/O to create virtual GPU instances. It handles physical-to-virtual interrupt routing, intercepts and emulates PCI configuration registers, and executes the scheduling algorithms that control access to GPU hardware.

2. Guest Component: NVIDIA vGPU Software Driver

• Role: Installed inside the guest VM operating system (Windows or Linux).
• Functionality: Presents itself to the guest OS as a standard NVIDIA graphics and compute adapter. It delivers full application binary interface (ABI) compatibility with CUDA, cuDNN, TensorRT, DirectX, Vulkan, and OpenGL. Applications running in the VM execute without code modification.

3. Licensing Infrastructure: NVIDIA License System (NLS)

To activate full compute performance and multi-monitor resolution features, guest drivers must check out a license token from the NVIDIA License System (NLS) upon booting.

+---------------------------------------------------------------------------------------------------+
|                         NVIDIA LICENSE SYSTEM (NLS) ARCHITECTURE                                  |
|                                                                                                   |
|  1. CLOUD LICENSE SERVER (CLS)                    2. DELEGATED LICENSE SERVER (DLS)               |
|  ┌────────────────────────────────────────┐       ┌────────────────────────────────────────┐      │
|  │ NVIDIA Hosted Cloud Service            │       │ On-Premises Local Virtual Appliance    │      │
|  │ - Internet-connected environments      │       │ - Air-gapped & sovereign data centers  │      │
|  │ - Fully managed high availability      │       │ - High-security enterprise enclaves    │      │
|  │ - Direct token leasing over HTTPS      │       │ - Synchronizes periodically or offline │      │
|  └────────────────────────────────────────┘       └────────────────────────────────────────┘      │
|                       ▲                                               ▲                           │
|                       │                                               │                           │
|                       └──────────── Client Config Token ──────────────┘                           │
|                                     (client_configuration_token.tok)                              │
|                                                       ▲                                           │
|                                                       │ (Lease Request on Boot)                   │
|                                            ┌──────────┴──────────┐                                │
|                                            │ Guest VM vGPU Driver│                                │
|                                            └─────────────────────┘                                │
+---------------------------------------------------------------------------------------------------+

• Cloud License Server (CLS): Hosted and managed by NVIDIA in the cloud. Client VMs with outbound internet connectivity lease license tokens directly over HTTPS.
• Delegated License Server (DLS): A containerized virtual appliance installed locally on customer premises. Designed for air-gapped, defense, financial, or sovereign environments where VMs have no external internet access. The DLS downloads an encrypted license package from the NVIDIA Licensing Portal and serves leases locally across the internal subnet.
• Client Configuration Token (client_configuration_token.tok): An encrypted token file generated from the CLS/DLS portal and placed inside the guest VM filesystem (/etc/nvidia/ClientConfigToken/ on Linux). It defines the IP address, port, and security certificates required for the guest driver to acquire its license automatically at startup.

---

2. NVIDIA vGPU Software Editions & Workload Profiles

NVIDIA packages vGPU software into distinct editions tailored to specific enterprise operational requirements:

+---------------------------------------------------------------------------------------------------+
|                             NVIDIA vGPU SOFTWARE PRODUCT EDITIONS                                 |
+----------------------------+-----------------------------+----------------------------------------+
| EDITION                    | TARGET WORKLOADS            | KEY FEATURES & CAPABILITIES            |
+----------------------------+-----------------------------+----------------------------------------+
| **NVIDIA Virtual PC**      | Knowledge Workers,          | - Supports up to four 4K displays      |
| **(vPC)**                  | VDI Desktops, Web Browsing, | - NVENC/NVDEC hardware video decode    |
|                            | Office Productivity         | - 1GB–2GB VRAM Profiles                |
|                            |                             | - CUDA Compute DISABLED                |
+----------------------------+-----------------------------+----------------------------------------+
| **NVIDIA RTX Virtual**     | CAD / CAE Engineers,        | - Full ISV Certification (AutoCAD, etc)|
| **Workstation (vWS)**      | 3D Animators, Visual FX,    | - Full NVIDIA RTX & OptiX Ray Tracing  |
|                            | Professional Graphics       | - Up to four 5K / two 8K displays      |
|                            |                             | - CUDA Compute & Graphics ENABLED      |
+----------------------------+-----------------------------+----------------------------------------+
| **NVIDIA Virtual Compute** | Headless AI Training,       | - Headless Operation (No Display Stack)|
| **Server (vCS) /**         | Deep Learning Inference,    | - Full Uncapped CUDA, cuDNN, TensorRT  |
| **AI Enterprise (NVAIE)**  | HPC, RAPIDS Data Science    | - GPU Clustering & Multi-GPU VM Mapping|
|                            |                             | - Optimized for Docker / K8s in VMs    |
+----------------------------+-----------------------------+----------------------------------------+

vGPU Profile Naming Taxonomy

Every vGPU profile follows a standardized naming convention: <Physical GPU Model>-<VRAM in GB><License Code>.

  Example: A100-4C  ──► NVIDIA A100 GPU, 4 GB Framebuffer, 'C' Series (vCS / AI Compute)
  Example: A40-8Q   ──► NVIDIA A40 GPU, 8 GB Framebuffer, 'Q' Series (vWS / Workstation)
  Example: A40-2B   ──► NVIDIA A40 GPU, 2 GB Framebuffer, 'B' Series (vPC / Virtual PC)
  Example: A16-4A   ──► NVIDIA A16 GPU, 4 GB Framebuffer, 'A' Series (Virtual Apps)

---

3. GPU Virtualization Sharing Models

When virtualizing GPUs, hypervisors employ three primary resource-sharing models:

+---------------------------------------------------------------------------------------------------+
|                             vGPU RESOURCE ALLOCATION MODELS                                       |
|                                                                                                   |
|  1. TIME-SLICED (Best Effort)       2. FIXED-SHARE (Equal Time)    3. MIG-BACKED vGPU (Hardware)  |
|  ┌───────────────────────────┐      ┌───────────────────────────┐  ┌───────┬───────┬───────┬───────┐│
|  │ VM 1: Takes available SMs │      │ VM 1: Exactly 25% Time    │  │ VM 1  │ VM 2  │ VM 3  │ VM 4  ││
|  ├───────────────────────────┤      ├───────────────────────────┤  │ (MIG) │ (MIG) │ (MIG) │ (MIG) ││
|  │ VM 2: Takes remaining SMs │      │ VM 2: Exactly 25% Time    │  │ Slice │ Slice │ Slice │ Slice ││
|  ├───────────────────────────┤      ├───────────────────────────┤  ├───────┴───────┴───────┴───────┤│
|  │ VM 3: Waits in Queue      │      │ VM 3: Exactly 25% Time    │  │ Dedicated SMs, L2, & HBM Chs  ││
|  └───────────────────────────┘      └───────────────────────────┘  └───────────────────────────────┘│
|  * Dynamic SM multiplexing          * Rigid round-robin slices     * True silicon-level partitioning│
|  * High burst capacity              * Deterministic execution time * Zero scheduling jitter         │
+---------------------------------------------------------------------------------------------------+

1. Time-Sliced vGPU (Best-Effort & Equal-Share)

• Mechanism: Multiple VMs share the same physical Streaming Multiprocessors over time. The host vGPU Manager context-switches between VMs on millisecond intervals.
• Best-Effort Scheduler: If only one VM is actively running a workload, it consumes 100% of the physical GPU compute cycles. When other VMs become active, compute time is divided dynamically.
• Equal-Share Scheduler: Divides execution time into fixed, equal time quantums per active VM, preventing a single runaway VM from starving others.
• Trade-off: High density for bursty virtual desktop infrastructure (VDI), but introduces latency jitter and context-switching overhead unsuitable for strict real-time AI inference SLAs.

2. MIG-Backed vGPU Profiles (Hardware Slicing + Hypervisor Management)

Introduced with Ampere and Hopper GPUs, MIG-backed vGPU represents the convergence of silicon-level partitioning and hypervisor virtualization.

• Mechanism: The hypervisor host vGPU Manager first enables MIG mode on the physical GPU and instantiates hardware GPU Instances (e.g., A100-1-10C, A100-2-20C, A100-3-40C, A100-7-80C).
• Assignment: Each hardware MIG slice is bound directly to a virtual machine via the mediated device (mdev) interface.
• Operational Advantages:
  1. Zero Time-Slicing Overhead: Because each VM executes on physically dedicated SMs, L2 cache lines, and HBM memory channels, there is zero context switching or scheduling jitter.
  2. Hypervisor Enterprise Features: VMs retain enterprise hypervisor capabilities, including VM lifecycle automation, high availability (vSphere HA), centralized backup, and network virtualization (NSX).
  3. Multi-Tenant Security: Provides two complementary layers of isolation: hardware-level silicon memory boundaries and hypervisor-level CPU/memory virtual machine boundaries.