5.2 Space-Efficient Redirect-on-Write (ROW) Snapshots and Zero-Byte Clones

5.2 Space-Efficient Redirect-on-Write (ROW) Snapshots and Zero-Byte Clones

[!IMPORTANT] The Architectural Difference: Traditional virtualization platforms suffer from severe snapshot performance penalties, "snapshot bloat," and excruciating consolidation times. Nutanix AOS fundamentally solves this problem by decoupling logical virtual disks from physical disk extents using a distributed metadata pointer architecture based on Redirect-on-Write (ROW). Snapshots and clones execute instantaneously with zero upfront capacity and zero write performance penalty.

Virtual machine snapshots are essential for routine administrative workflows: patching operating systems, updating enterprise applications, performing database rollbacks, and facilitating continuous backup schedules. However, in legacy shared storage arrays and traditional hypervisors, snapshots were treated as temporary, high-risk objects that degraded storage performance and risked filling datastores if left uncommitted. Nutanix transforms snapshots and clones into enterprise-grade, high-performance operational primitives.

---

Redirect-on-Write (ROW) vs. Traditional Copy-on-Write (COW)

To understand why Nutanix snapshots impose zero ongoing performance degradation, one must compare the write mechanics of legacy Copy-on-Write (COW) with modern Redirect-on-Write (ROW).

TRADITIONAL COPY-ON-WRITE (COW) - 3-STEP I/O PENALTY:
Incoming Write to Block A
   1. Read original Block A from base disk  ──> [ Read Penalty ]
   2. Write original Block A to delta file  ──> [ Write Penalty ]
   3. Overwrite Block A on base disk        ──> [ Write Penalty ]
   ===> Result: 3 I/O operations for every single guest write! (Severe latency spike)

NUTANIX REDIRECT-ON-WRITE (ROW) - ZERO PENALTY:
Incoming Write to Block A
   1. Write new Block A' to fresh, free storage extent ──> [ Single Standard Write ]
   2. Update Cassandra metadata pointer for Active vDisk ──> [ Pointer Update in Memory ]
   (Snapshot pointer still points to original immutable Block A)
   ===> Result: Exactly 1 write operation! Identical to normal storage I/O.

The Legacy Copy-on-Write (COW) Penalty

In traditional storage systems and standard hypervisors (such as classic VMDK delta snapshots):

1. When a snapshot is created, the base virtual disk is locked.
2. When an application issues a write to an existing block (e.g., Block A), the storage subsystem cannot simply overwrite Block A, because the snapshot must preserve Block A's original state.
3. The hypervisor must first read the original data from Block A.
4. It must write that original data into a dedicated snapshot delta file or snapshot reserve volume.
5. Only after the copy succeeds can it finally write the new modified data into Block A.

This creates a 3-step I/O penalty (1 Read + 2 Writes) for every modified sector. As more snapshots are added to a chain, disk I/O queues become saturated, write latencies spike exponentially, and the process of deleting or "committing" the snapshot requires hours of intensive disk I/O that frequently stuns the running virtual machine.

Nutanix Redirect-on-Write (ROW) Architecture

In the Nutanix Distributed Storage Fabric, data blocks are immutable once written to disk extents. AOS does not overwrite existing data in place:

1. When a snapshot is taken, the current state of the virtual disk (vDisk) is preserved instantaneously simply by locking its metadata pointer tree.
2. When the guest VM issues a write to Block A, the local Stargate process allocates a brand-new, free extent location (Block A') in flash or OpLog and writes the new data directly.
3. Stargate informs Cassandra to update the metadata mapping table: the active vDisk now points to Block A', while the snapshot metadata record continues pointing to original Block A.
4. The write operation requires exactly 1 physical write—the exact same amount of work as writing to a non-snapshotted disk. There is zero read penalty, zero copy penalty, and zero virtual machine stunning.

---

Cassandra Distributed Metadata Pointer Trees

At the heart of Nutanix space efficiency is Cassandra, the distributed, key-value metadata store running on every CVM:

+-----------------------------------------------------------------------------------+
|                         Cassandra Metadata Extent Mapping                         |
+-----------------------------------------------------------------------------------+
| Logical vDisk Block | Referencing Entity      | Physical Extent ID on SSD/HDD     |
|---------------------|-------------------------|-----------------------------------|
| LBA 0 - 1 MB        | Base VM + Snapshot 1    | Extent_101 [Ref Count: 2]         |
| LBA 32 - 64 KB      | Base VM + Snapshot 1    | Extent_102 [Ref Count: 2]         |
| LBA 64 - 96 KB      | Snapshot 1 (Old Data)   | Extent_103 [Ref Count: 1]         |
| LBA 64 - 96 KB      | Base VM (Modified Data) | Extent_104 [Ref Count: 1] (New!)  |
+-----------------------------------------------------------------------------------+

• Every vDisk in Nutanix is represented in Cassandra as a tree of pointers that map Logical Block Addresses (LBAs) requested by the guest OS to physical Extent IDs and Extent Slices across cluster drives.
• Taking a snapshot does not duplicate a single byte of application data on physical storage. Instead, it creates a lightweight snapshot object in Cassandra that duplicates only the root pointer references.
• Because Cassandra operations are distributed and execute in CVM RAM and fast metadata SSD logs, snapshot creation completes in tens of milliseconds, regardless of whether the virtual machine disk is 10 GB or 50 TB.

---

Zero-Byte Clones: Instantaneous VM & vDisk Duplication

Nutanix leverages its Redirect-on-Write metadata architecture to deliver Zero-Byte Clones for both virtual machines and independent vDisks.

SOURCE VM:                      CLONED VM 1 (Test/Dev):        CLONED VM 2 (QA):
[ vDisk Metadata Tree ]          [ vDisk Metadata Tree ]        [ vDisk Metadata Tree ]
         │                                │                              │
         ├────────────────────────────────┼──────────────────────────────┤
         ▼                                ▼                              ▼
[ Extent 1: OS Kernel ]          [ Extent 2: App Binaries ]     [ Extent 3: Base DB ]
       (100% Shared Physical Data Extents - ZERO Additional Disk Consumption)

Cloning Mechanics

When an administrator clones a VM in Prism Element or Prism Central:

1. The Acropolis master instructs Cassandra to clone the vDisk metadata pointer table of the source VM.
2. A new set of VM configuration parameters (UUID, MAC address, virtual network interface) is registered in the hypervisor.
3. The new clone instantly references the exact same physical storage extents as the source parent.
4. Physical Capacity Consumed at Creation: Exactly 0 bytes.
5. Creation Time: Near-instantaneous (typically under 2 to 5 seconds per VM).

Branching and Write Independence

The clone is completely independent of the parent VM from the moment of creation:

• If the source VM modifies data, only the source VM allocates new extents via Redirect-on-Write. The clone continues referencing the original extents.
• If the clone modifies data, it writes new private extents. The parent VM remains completely unaffected.
• Multiple clones can be created from a single parent or even from other clones, forming complex branching trees without performance degradation.

Enterprise Use Cases for Zero-Byte Clones

• Virtual Desktop Infrastructure (VDI): Instantly provisioning hundreds of non-persistent or persistent pooled desktop virtual machines from a single golden base image.
• Test/Development Sandboxes: Refreshing a multi-terabyte production Oracle or SQL Server database into development environments in seconds without requiring terabytes of auxiliary storage.
• Disaster Recovery and Patch Testing: Verifying OS patches, service packs, or security updates against a live copy of a mission-critical workload prior to production rollouts.

---

Snapshot Expiration and Curator Background Garbage Collection

In legacy storage, deleting a snapshot from a long chain required "snapshot consolidation"—a risky, CPU-intensive process that read delta files, merged them into base disks, and frequently caused guest VM time-outs or filesystem corruption. In Nutanix AOS, snapshot deletion is completely non-disruptive.

1. SNAPSHOT DELETION:
   Administrator / Schedule deletes Snapshot ──> Cassandra marks snapshot metadata DELETED
                                                 (No disk I/O in the VM data path!)

2. CURATOR MAPREDUCE SWEEP:
   Curator background job scans Cassandra ───> Checks Reference Count on each extent
   - Extent A: Ref Count = 2 (Shared by VM & Snap 2) ──> Keep extent
   - Extent B: Ref Count = 0 (Referenced by NO ONE!)  ──> Flag for Garbage Collection

3. STARGATE RECLAMATION:
   Stargate receives GC task list ───> Unlinks physical blocks from SSD/HDD
   ===> Raw capacity returned to the cluster storage pool transparently!

1. The Deletion Phase

When an administrator deletes a snapshot or when a scheduled snapshot reaches its configured expiration timestamp:

• AOS does not execute foreground block manipulations or disk consolidation.
• Cassandra simply flags the snapshot record as tombstoned/deleted and removes its pointers from the active metadata tree.
• The guest virtual machine experiences zero I/O latency, zero lockup, and zero risk of corruption.

2. Reference Counting and Curator Analysis

Every physical extent tracked in the Cassandra metadata store maintains an internal reference count indicating how many active vDisks, snapshots, or clones are currently pointing to it:

• Reference Count >= 1: The extent is actively utilized and cannot be touched.
• Reference Count = 0: The extent is orphaned (the snapshot that owned it was deleted, and the active vDisk has since overwritten that block).

Periodically, the distributed Curator service initiates background MapReduce sweeps across the cluster to evaluate extent reference counts.

3. Physical Storage Reclamation by Stargate

Once Curator identifies extents with a reference count of zero:

1. Curator compiles a list of deallocated extent IDs and dispatches cleanup instructions to the Stargate daemons managing the underlying physical disks.
2. Stargate unlinks the physical blocks from the storage controller tables, zeroes the allocation maps, and returns the physical sectors to the available storage pool.
3. The freed space is immediately reflected in Prism storage charts as available cluster capacity.

---

Summary Comparison: COW vs. Nutanix ROW