5.3 Protection Domains, Consistency Groups, and Disaster Recovery Replication

5.3 Protection Domains, Consistency Groups, and Disaster Recovery Replication

[!NOTE] Business Continuity on Nutanix: Enterprise business continuity demands flexible recovery options that balance operational costs, wide-area network (WAN) bandwidth, and recovery metrics. Nutanix native data protection provides built-in disaster recovery (DR) without requiring third-party replication appliances or specialized SAN storage mirrors, scaling from multi-hour asynchronous snapshots down to zero-RPO synchronous metro clusters.

Disaster Recovery (DR) in Nutanix AOS is managed through policy-driven automation. By defining what to protect, how frequently to capture point-in-time recovery points, and where to replicate those recovery points across geographic boundaries, organizations can seamlessly satisfy demanding Service Level Agreements (SLAs).

---

Protection Domains (PD): Architecture and Mechanics

A Protection Domain (PD) is a logical container defined in Prism Element that encompasses a specific group of virtual machines and/or storage Volume Groups that share common recovery requirements.

+-----------------------------------------------------------------------------------+
|                         PROTECTION DOMAIN: "Production-Apps"                       |
+-----------------------------------------------------------------------------------+
| Entities:                                                                         |
|   - Web-VM-01 (AHV VM)                                                           |
|   - App-VM-01 (AHV VM)                                                           |
|   - DB-Volume-Group (Nutanix Volume Group via iSCSI)                              |
+-----------------------------------------------------------------------------------+
| Snapshot & Replication Schedule:                                                  |
|   - Local Schedule:  Every 1 hour, retain last 24 snapshots                       |
|   - Remote Schedule: Replicate to 'DR-Cluster-Secondary' every 4 hours, retain 14 |
+-----------------------------------------------------------------------------------+

Protection Domain Operational States

At any given time, a Protection Domain exists in one of two states across paired clusters:

1. Active State: Resides on the primary production cluster. The virtual machines are registered, powered on, and serving live I/O. Snapshot schedules actively execute, and outgoing replication streams delta extents to the secondary cluster.
2. Inactive State: Resides on the target disaster recovery cluster. The Protection Domain holds replicated point-in-time snapshots and VM configuration metadata, but the virtual machines are not registered or running in the hypervisor inventory until a failover is initiated.

Migration vs. Unplanned Failover

Nutanix provides two distinct failover workflows within Protection Domains:

• Planned Migration (Clean Migration): Executed during scheduled datacenter maintenance, hurricane warnings, or planned power outages.
  1. The administrator initiates migration from the primary cluster.
  2. Production VMs on the primary site are cleanly powered off.
  3. A final, zero-delta snapshot is captured and replicated across the WAN to synchronize the very last I/O writes.
  4. The primary Protection Domain transitions to Inactive, while the secondary Protection Domain transitions to Active.
  5. The target cluster automatically registers the VMs in the hypervisor inventory and powers them on, resulting in zero data loss (RPO = 0).
• Disaster Recovery (Unplanned Failover): Executed when the primary datacenter suffers a catastrophic event (such as total power failure or flood) and is unreachable.
  1. The administrator logs into the secondary disaster recovery cluster.
  2. The administrator selects the inactive Protection Domain and clicks Activate.
  3. The secondary cluster reconstructs the VMs from the most recently received replicated snapshot.
  4. VMs are registered and powered on. The data loss corresponds to the time elapsed since the last completed replication (governed by the configured RPO).

---

Consistency Groups (CG): Preserving Multi-VM Application Integrity

A Consistency Group (CG) is a subset of virtual machines and volume groups configured inside a Protection Domain that must be captured simultaneously at the exact same instant.

PROTECTION DOMAIN: E-Commerce-Suite
│
├── CONSISTENCY GROUP: Database-Cluster (Atomically Snapshotted Together)
│     ├── SQL-Node-01 (Database Master)
│     └── SQL-Node-02 (Database Replica)
│
└── Standalone VM: Reporting-Server (Snapshotted Independently)

Why Consistency Groups are Vital

Consider a distributed, multi-tier enterprise application where an application server processes customer transactions and writes them to a backend SQL database. If the application server is snapshotted at 14:00:00 and the database server is snapshotted at 14:00:05, those five seconds of discrepancy mean the restored database will contain records that the application server's restored state has no memory of, or vice versa. Consistency Groups eliminate this discrepancy by triggering atomic, synchronized snapshot creation across all member VMs.

Crash-Consistent vs. Application-Consistent Snapshots

[!IMPORTANT] Nutanix Guest Tools (NGT) and VSS: To achieve true application-consistent snapshots for transactional applications (such as Microsoft SQL Server, Microsoft Exchange, or Oracle), NGT must be installed inside the guest VM. When a scheduled snapshot fires, the CVM communicates via NGT to engage the VSS hardware provider, temporarily pausing application writes and flushing database logs before the snapshot is stamped.

---

The Nutanix DR Replication Spectrum: Async, NearSync, and Synchronous

Disaster recovery strategies must balance the strictness of the Recovery Point Objective (RPO)—the maximum acceptable amount of data loss measured in time—against network bandwidth availability, geographic distance, and infrastructure cost.

+-----------------------------------------------------------------------------------------+
|                               Nutanix Replication Spectrum                              |
+-----------------------------------------------------------------------------------------+
| Replication Type   | Supported RPO          | Network Latency (RTT) | Target Distance   |
|--------------------|------------------------|-----------------------|-------------------|
| Asynchronous (Async)| >= 60 Minutes          | Any WAN Latency       | Global / Unlimited|
| NearSync           | 20 Seconds to 15 Mins  | <= 80 ms RTT          | Regional / Cross-Co|
| Synchronous (Metro)| 0 (Zero Data Loss)     | <= 5 ms RTT           | Metro (< 50-100 km)|
+-----------------------------------------------------------------------------------------+

1. Asynchronous Replication (RPO >= 60 minutes)

• Operational Mechanics: Standard scheduled snapshot replication. When the scheduled window triggers, AOS takes an in-memory ROW snapshot. Cassandra compares the extent mapping of the new snapshot against the previously replicated snapshot to generate a list of modified blocks (delta extents).
• Transfer: Stargate compresses the delta extents and transmits them over standard IP WAN links to the remote cluster, where they are reassembled into a remote snapshot.
• Network Tolerance: Resilient across high-latency, lower-bandwidth wide-area connections with no strict latency ceiling.

2. NearSync Replication (RPO = 20 seconds to 15 minutes)

NearSync fills the critical gap between hourly asynchronous snapshots and high-cost synchronous metro clustering:

• Light-Weight Snapshots (LWS): NearSync does not use traditional heavy snapshot metadata trees. Instead, it utilizes Light-Weight Snapshots (LWS) that track write markers in CVM memory and OpLog.
• Continuous Streaming: Modified extents are continuously synchronized across the network to the secondary cluster every few seconds.
• Automatic State Machine (Fallback and Promotion): NearSync requires a network round-trip time (RTT) of 80 ms or less. If network bandwidth degrades or network latency spikes cause the replication queue to exceed the target RPO, AOS automatically and nondisruptively falls back to standard Asynchronous replication. Once WAN conditions stabilize, the system automatically catches up and promotes itself back into NearSync mode without administrative intervention.

3. Synchronous Replication / Metro Availability (RPO = 0)

Synchronous replication, often implemented as Nutanix Metro Availability, guarantees zero data loss:

SYNCHRONOUS WRITE DATAPATH (RPO = 0):
Guest Write ──> Host Hypervisor ──> Local CVM Stargate
                                         │
                  ┌──────────────────────┴──────────────────────┐
                  ▼                                             ▼
      Commit to Local OpLog/SSD                     Transmit via Metro Network
                  │                                             │
                  │                                             ▼
                  │                                 Commit to Remote OpLog/SSD
                  │                                             │
                  │                                  Acknowledge Remote Write
                  │                                             │
                  └──────────────────────┬──────────────────────┘
                                         ▼
                  Both Sites Acknowledged! Stargate completes I/O
                                         │
                                         ▼
                  Acknowledge Write Completion to Guest Hypervisor

• Write Pipeline: When a VM issues a write, the local CVM simultaneously writes to its local OpLog/SSD and transmits the payload over the high-speed link to the remote CVM. The hypervisor does not receive write confirmation until both physical clusters acknowledge the write to non-volatile storage.
• Strict Network Prerequisites:
  • Maximum network round-trip time (RTT) of 5 milliseconds or less (typically requiring dedicated dark fiber within a 50 to 100 km radius).
  • Minimum bandwidth of 1 Gbps (10 Gbps strongly recommended for heavy workloads).
• The Metro Availability Witness:
  • In a two-site synchronous deployment, if network connectivity between Site A and Site B is severed, both sites might assume the other has failed. If both sites attempt to serve active writes to the same virtual machines, a catastrophic split-brain state occurs, causing massive data corruption.
  • To prevent this, Nutanix deploys an independent Metro Witness (a lightweight virtual machine residing in an independent third failure domain, such as AWS, Azure, or a third corporate office).
  • If inter-site communication fails, each cluster queries the Witness. The Witness grants write authority to only one cluster, cleanly isolating the other to prevent split-brain.

---

Remote Sites and Snapshot Retention Policies

When configuring replication within a Protection Domain, administrators configure distinct local and remote retention rules:

LOCAL RETENTION (Primary Site):               REMOTE RETENTION (DR Site):
Retain last 12 hourly snapshots               Retain last 48 hourly snapshots
Retain last 7 daily snapshots                 Retain last 30 daily snapshots
Retain last 4 weekly snapshots                Retain last 12 monthly snapshots
(Focus: Rapid local operational recovery)     (Focus: Long-term DR & compliance archive)

Snapshot Roll-Up Retention

AOS supports automated snapshot roll-up. For example, an administrator can specify: take a snapshot every 1 hour, keep 24 hourly snapshots, keep 7 daily snapshots, and keep 4 weekly snapshots. At the end of each day, AOS automatically designates one snapshot as the daily checkpoint and deletes redundant hourly snapshots, conserving cluster capacity while maintaining historical depth.

Network Bandwidth Throttling

To prevent background disaster recovery replication from saturating wide-area production circuits, administrators can configure Network Bandwidth Throttling policies between remote site pairings in Prism Element:

• Bandwidth limits can be defined in Megabits per second (Mbps).
• Throttling schedules can be enforced dynamically based on time of day (e.g., limit DR replication traffic to 50 Mbps between 08:00 and 18:00 on business days, but allow uncapped 1 Gbps replication overnight and on weekends).