8.1 Multi-Instance GPU (MIG) Architecture & Profiles

8.1 Multi-Instance GPU (MIG) Architecture & Profiles

Core Principle: In modern enterprise AI infrastructure, monolithic modern GPUs—such as the NVIDIA A100 (80GB) and NVIDIA H100 (80GB)—possess massive compute throughput and memory bandwidth that often exceed the requirements of individual inference microservices, interactive development sessions, or lightweight fine-tuning jobs. Multi-Instance GPU (MIG) addresses this underutilization by partitioning a single physical GPU into up to seven independent, hardware-isolated GPU instances, each with dedicated compute, memory, and interconnect resources.

---

1. The Multi-Instance GPU (MIG) Paradigm

Prior to the introduction of MIG in the NVIDIA Ampere architecture (GA100) and its continued advancement in Hopper (GH100), sharing a GPU across multiple concurrent workloads relied on software-level time-slicing or the CUDA Multi-Process Service (MPS). While software sharing allows multiple processes to submit kernels to the same GPU, it suffers from critical limitations:

• Noisy Neighbor Contention: Processes compete dynamically for Streaming Multiprocessors (SMs), shared L2 cache capacity, and DRAM memory bandwidth. A memory-intensive workload can degrade the latency and throughput of co-located services.
• Lack of Fault Isolation: A memory fault, out-of-memory (OOM) error, or unhandled CUDA exception in one process can crash or corrupt the shared CUDA context, terminating all co-located workloads.
• Unpredictable Quality of Service (QoS): Latency-sensitive inference workloads cannot guarantee Service Level Agreements (SLAs) when running alongside unpredictable batch processes.

MIG fundamentally transforms GPU sharing by implementing silicon-level hardware partitioning. Instead of time-sharing execution cycles, the GPU silicon is physically sliced into discrete instances that operate in parallel with complete independence.

+---------------------------------------------------------------------------------------------------+
|                         MONOLITHIC GPU vs. MULTI-INSTANCE GPU (MIG)                              |
|                                                                                                   |
|  UNPARTITIONED GPU (Single Tenant / Shared)        MIG PARTITIONED GPU (Up to 7 Instances)        |
|  ┌───────────────────────────────────────────┐    ┌───────┬───────┬───────┬───────┬───────┬───────┐│
|  │ Total Compute: 100% SMs (Shared)          │    │ MIG 1 │ MIG 2 │ MIG 3 │ MIG 4 │ MIG 5 │ MIG 6 ││
|  ├───────────────────────────────────────────┤    ├───────┼───────┼───────┼───────┼───────┼───────┤│
|  │ Shared L2 Cache (Contention Risk)         │    │ L2-S0 │ L2-S1 │ L2-S2 │ L2-S3 │ L2-S4 │ L2-S5 ││
|  ├───────────────────────────────────────────┤    ├───────┼───────┼───────┼───────┼───────┼───────┤│
|  │ Shared HBM Memory Channels (No QoS)       │    │ HBM-0 │ HBM-1 │ HBM-2 │ HBM-3 │ HBM-4 │ HBM-5 ││
|  └───────────────────────────────────────────┘    └───────┴───────┴───────┴───────┴───────┴───────┘│
|  * Software Time-Slicing                          * True Hardware Silicon Slicing                 │
|  * Vulnerable to Noisy Neighbors                  * Deterministic Latency & Guaranteed QoS        │
|  * Single Point of Failure (Crash Impact)         * Complete Hardware Fault Isolation             │
+---------------------------------------------------------------------------------------------------+

Supported Hardware Platforms

MIG is natively integrated into enterprise-class NVIDIA data center GPUs:

• Ampere Architecture: NVIDIA A100 SXM4 (40GB & 80GB), NVIDIA A100 PCIe (40GB & 80GB), NVIDIA A30 (24GB HBM2, supports up to 4 instances).
• Hopper Architecture: NVIDIA H100 SXM5 (80GB HBM3), NVIDIA H100 PCIe (80GB HBM2e), NVIDIA H200 (141GB HBM3e).
• Blackwell Architecture: NVIDIA B200 and GB200 series (supporting multi-instance enterprise partitioning).

---

2. Silicon-Level Hardware Isolation Mechanics

To understand why MIG delivers deterministic performance, we must examine how the underlying GPU silicon resources are partitioned across the compute, memory, and interconnect subsystems.

                    MIG HARDWARE SILICON ISOLATION SUBSYSTEMS

  ┌────────────────────────────────────────────────────────────────────────┐
  │               PHYSICAL STREAMING MULTIPROCESSORS (SMs)                 │
  │  Grouped into Graphics Processing Clusters (GPCs) - Dedicated per Slice│
  ├────────────────────────────────────────────────────────────────────────┤
  │                    ON-CHIP L2 CACHE PARTITIONS                         │
  │  Physical SRAM lines carved into isolated addressable slices           │
  ├────────────────────────────────────────────────────────────────────────┤
  │               MEMORY CROSSBAR & MEMORY CONTROLLERS                     │
  │  Dedicated crossbar routing ports & isolated physical DRAM channels    │
  ├────────────────────────────────────────────────────────────────────────┤
  │                  DMA COPY ENGINES & MMU ENGINES                        │
  │  Independent Page Table directories, fault loggers, and DMA queues     │
  └────────────────────────────────────────────────────────────────────────┘

1. Compute Subsystem Partitioning (SMs & GPCs)

In modern NVIDIA microarchitectures, Streaming Multiprocessors (SMs) are organized into Graphics Processing Clusters (GPCs). On an A100 GPU (which has 7 active GPCs in its data center configuration), MIG allocates physical GPCs directly to instances:

• A 1-slice instance receives 1 full GPC (~14 SMs on A100, ~16–18 SMs on H100).
• A 2-slice instance receives 2 full GPCs (~28 SMs on A100).
• A 3-slice instance receives 3 full GPCs (~42 SMs on A100).
• A 7-slice instance receives all 7 active GPCs (the full GPU). Because SMs are physically mapped to instances at initialization, there is zero instruction pipeline sharing or warp scheduling contention between instances.

2. On-Chip L2 Cache Partitioning

The monolithic high-speed L2 cache (40 MB on both the A100 40GB and the A100 80GB — the extra capacity in the 80GB part is HBM, not cache — and 50 MB on the H100 SXM5) is physically carved into dedicated slices. Each MIG instance is assigned a private segment of the L2 cache corresponding directly to its compute size:

• A 1g.10gb instance receives exactly 1/8th or 1/7th of the physical L2 cache lines.
• A tenant cannot evict, overwrite, or throttle the cache lines of an adjacent tenant.

3. Memory Subsystem & DRAM Controller Isolation

High-Bandwidth Memory (HBM2e / HBM3) communicates with the GPU core via an ultra-wide memory crossbar connected to multiple physical memory controllers. MIG assigns dedicated memory crossbar ports and isolated DRAM channels to each instance:

• A 1g.10gb profile on an 80GB GPU receives dedicated access to 10GB of physical HBM with guaranteed memory bandwidth (proportional to its slice width, ~240 GB/s to 480 GB/s).
• Even if one instance executes continuous uncoalesced memory reads that saturate its assigned memory channel, adjacent instances maintain 100% of their rated memory bandwidth without experiencing latency spikes.

4. DMA Engines, MMUs, and Fault Isolation

Each MIG instance is equipped with its own dedicated DMA Copy Engines, Memory Management Unit (MMU) page table directories, and fault logging registers:

• Each instance runs its own isolated address space; memory addresses cannot overlap or cross instance boundaries.
• If an application inside Instance A triggers an out-of-memory (OOM) error, a CUDA illegal memory address trap, or a kernel timeout, the GPU driver resets only Instance A. Instances B through G continue executing without interruption.

---

3. MIG Naming Conventions & Profile Specifications

NVIDIA standardized the naming structure for MIG profiles to explicitly declare the compute and memory resources assigned to each partition.

The MIG Profile Syntax

                     [Compute Slices]c . [GPU Slices]g . [Memory Size]gb
                     
  Example: 1g.10gb  ──► 1 GPU Slice (Compute + Memory), 10 GB HBM Memory
  Example: 2g.20gb  ──► 2 GPU Slices (Compute + Memory), 20 GB HBM Memory
  Example: 3g.40gb  ──► 3 GPU Slices (Compute + Memory), 40 GB HBM Memory
  Example: 1c.2g.20gb ─► 1 Compute Instance inside a 2-Slice GPU Instance (20GB)

When the compute slice count matches the GPU slice count (which is the standard configuration), the prefix 1c. is omitted, simplifying the notation to <N>g.<M>gb.

Standard MIG Profiles on 80GB Enterprise GPUs (A100 80GB / H100 80GB)

Valid Profile Combinations and Placement Rules

Because MIG partitions are physically mapped to contiguous hardware silicon blocks, instances must satisfy strict placement rules across the 7 available GPU slice slots:

• Slices cannot be fragmented arbitrarily across physical GPCs.
• A 3g.40gb instance occupies 4 physical slice slots internally (providing 3 compute slices and 40GB memory) and can only start at slice index 0 or index 4.
• A 4g.40gb instance occupies physical slice indices 0 through 3.

+---------------------------------------------------------------------------------------------------+
|                         EXAMPLE VALID 80GB GPU MIG COMBINATIONS                                   |
|                                                                                                   |
|  Combination A (Max Density):    [ 1g.10gb ][ 1g.10gb ][ 1g.10gb ][ 1g.10gb ][ 1g.10gb ][ 1g.10gb ][ 1g.10gb ] (7 Instances)│
|  Combination B (Balanced):       [   2g.20gb   ][   2g.20gb   ][   2g.20gb   ][ 1g.10gb ] (4 Instances)                │
|  Combination C (Dual Mid-Tier):  [      3g.40gb       ][      3g.40gb       ] (2 Instances - no slots left)           │
|  Combination D (Compute Heavy):  [        4g.40gb         ][      3g.40gb       ] (2 Instances)                       │
|  Combination E (Mixed Slicing):  [        4g.40gb         ][   2g.20gb   ][ 1g.10gb ] (3 Instances)                   │
+---------------------------------------------------------------------------------------------------+

---

4. GPU Instances (GI) vs. Compute Instances (CI)

The NVIDIA MIG architecture implements a clear separation between GPU Instances (GI) and Compute Instances (CI).

┌─────────────────────────────────────────────────────────────────────────────────┐
│                     PHYSICAL GPU ASIC (Ampere / Hopper)                         │
│                                                                                 │
│  ┌────────────────────────────────────────┐ ┌────────────────────────────────┐  │
│  │        GPU INSTANCE 0 (GI 0)           │ │     GPU INSTANCE 1 (GI 1)      │  │
│  │  ┌──────────────────────────────────┐  │ │  ┌──────────────────────────┐  │  │
│  │  │ Dedicated 40GB HBM & L2 Cache    │  │ │  │ Dedicated 40GB HBM & L2   │  │  │
│  │  ├──────────────────────────────────┤  │ │  ├──────────────────────────┤  │  │
│  │  │ COMPUTE INSTANCE 0 (CI 0)        │  │ │  │ COMPUTE INSTANCE 0 (CI 0)│  │  │
│  │  │ 2 GPCs (~28 SMs)                 │  │ │  │ 3 GPCs (~42 SMs)         │  │  │
│  │  ├──────────────────────────────────┤  │ │  └──────────────────────────┘  │  │
│  │  │ COMPUTE INSTANCE 1 (CI 1)        │  │ │                                │  │
│  │  │ 1 GPC (~14 SMs)                  │  │ │                                │  │
│  │  └──────────────────────────────────┘  │ │                                │  │
│  └────────────────────────────────────────┘ └────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────────────────┘

GPU Instance (GI)

• Definition: The primary architectural partition that carves out physical memory, memory bandwidth, L2 cache slices, and memory crossbar routing, along with a specified pool of SMs.
• Scope: Defines the security and memory boundary. Two separate GIs cannot access each other's memory under any circumstances.
• OS Representation: Each GI appears as a distinct physical device path under /dev/ and receives a unique GPU Instance ID and UUID.

Compute Instance (CI)

• Definition: A compute-only sub-partition created inside an existing GPU Instance.
• Scope: Multiple CIs within the same parent GI share the parent's memory capacity, L2 cache, and memory bandwidth, but have dedicated, non-overlapping SMs.
• Use Case: Collaborative workloads within the same security domain (e.g., an inference pipeline where an audio preprocessor and a language model run in separate execution contexts but share a common high-bandwidth memory buffer).

Architectural Comparison: GI vs. CI