3.4 Storage Policy-Based Management (SPBM), Fault Tolerance, & Space Efficiency

3.4 Storage Policy-Based Management (SPBM), Fault Tolerance, & Space Efficiency

Executive Overview: Storage Policy-Based Management (SPBM) is the declarative software abstraction layer that translates business availability and performance Service Level Agreements (SLAs) into underlying vSAN object layouts. Rather than binding virtual disks to rigid, static storage arrays, SPBM empowers administrators to dynamically dictate redundancy, failure tolerance methods, space reservations, and I/O throttling per-virtual machine or per-virtual disk. For the VCP-VCF exam, candidates must master the mathematical capacity multipliers of RAID-1 versus erasure coding (RAID-5/6), understand ESA's adaptive striping, navigate quorum voting logic, and master the 60-minute clomd component repair timer.

---

The SPBM Framework in VMware Cloud Foundation

In traditional storage architectures, virtual machine performance and protection are fixed at the LUN or volume level. If a LUN is configured with RAID-5, every virtual disk residing on that LUN is forced to use RAID-5. Moving a VM to a high-performance mirrored tier requires a disruptive or time-consuming Storage vMotion migration across physical LUNs.

The Object-Based vSAN Abstraction

Under vSAN, storage is managed at the granularity of individual software Objects. Every virtual machine consists of multiple distinct objects:

• VM Home Namespace: Stores the .vmx configuration file, .nvram, descriptor files, and log files.
• VMDK Objects: Corresponds directly to virtual machine hard disks.
• VM Swap Object: Created when the VM powers on to back unreserved virtual RAM.
• Snapshot Delta Objects: Point-in-time branch objects storing modified data blocks.

Declarative Policy Enforcement

Through SPBM, administrators create declarative rule sets. When an SPBM policy is applied to a virtual machine or a specific virtual disk:

1. The vSAN Cluster Level Object Manager (CLOM) queries cluster topology, available host counts, and free storage capacity.
2. CLOM calculates the required component layout (replicas, parity segments, witness tiebreakers).
3. The Distributed Object Manager (DOM) instantiates the components across physical drives on distinct hosts.
4. Dynamic Non-Disruptive Changes: If an administrator alters an existing policy (e.g., converting a 2 TB VMDK from RAID-1 Mirroring to RAID-5 Erasure Coding), vSAN instantiates the new component layout in the background, synchronizes data blocks, and seamlessly switches over active I/O with zero workload downtime.

---

Primary Level of Failures to Tolerate (PFTT / FTT)

The foundational SPBM parameter is the Primary Level of Failures to Tolerate (PFTT), commonly designated as FTT:

• FTT=0: Zero redundancy. The object consists of a single component. If the underlying host or drive fails, the virtual disk becomes immediately unavailable. Used exclusively for ephemeral test/dev workloads or stateless container pods.
• FTT=1: Tolerates the concurrent loss of 1 host, 1 drive, or 1 fault domain. This represents the enterprise standard baseline.
• FTT=2: Tolerates the concurrent loss of 2 hosts, 2 drives, or 2 fault domains.
• FTT=3: Tolerates the concurrent loss of 3 hosts, 3 drives, or 3 fault domains, reserved for ultra-critical financial or medical infrastructure.

---

Failure Tolerance Methods: RAID-1 Mirroring vs. RAID-5/6 Erasure Coding

Administrators configure FTT in conjunction with the Failure Tolerance Method: RAID-1 (Mirroring) or Erasure Coding (RAID-5/6).

┌────────────────────────────────────────────────────────────────────────┐
│                     SPBM FAILURE TOLERANCE METHODS                     │
├───────────────────────┬────────────────────────┬───────────────────────┤
│ RAID-1 (Mirroring)    │ RAID-5 (Erasure Code)  │ RAID-6 (Erasure Code) │
├───────────────────────┼────────────────────────┼───────────────────────┤
│ Full data copies      │ Distributed Parity     │ Dual Distributed Par. │
│ Fast write throughput │ High space efficiency  │ Extreme resilience    │
│ High storage overhead │ Low storage overhead   │ Lowest FTT=2 overhead │
└───────────────────────┴────────────────────────┴───────────────────────┘

1. RAID-1 (Mirroring)

RAID-1 writes identical copies of each data block across distinct hosts:

• Component Architecture: To satisfy quorum, an FTT=n RAID-1 object requires n + 1 data replicas plus n witness components distributed across at least 2n + 1 hosts.
  • FTT=1 RAID-1: 2 Data Replicas + 1 Witness Component = 3 components across min. 3 hosts.
  • FTT=2 RAID-1: 3 Data Replicas + 2 Witness Components = 5 components across min. 5 hosts.
  • FTT=3 RAID-1: 4 Data Replicas + 3 Witness Components = 7 components across min. 7 hosts.
• Capacity Calculation Formula: Raw Storage Consumed = VMDK Size * (n + 1)
  • For FTT=1, the capacity multiplier is 2.0x (100% overhead). A 500 GB VMDK consumes 1,000 GB of raw storage.
  • For FTT=2, the capacity multiplier is 3.0x (200% overhead). A 500 GB VMDK consumes 1,500 GB of raw storage.
  • For FTT=3, the capacity multiplier is 4.0x (300% overhead). A 500 GB VMDK consumes 2,000 GB of raw storage.
• Workload Profile: Best suited for random write-intensive workloads (database write-ahead logs, transaction journals) where raw IOPS and minimal write path processing are critical.

2. RAID-5 Erasure Coding (OSA vs. ESA Adaptive)

Erasure coding breaks data into fragments and calculates distributed parity blocks. It satisfies FTT=1 while drastically reducing raw capacity consumption:

• vSAN OSA Fixed 3+1 Scheme: In legacy OSA, RAID-5 utilizes a rigid 3+1 parity stripe (3 data components + 1 parity component):
  • Requires a strict minimum of 4 hosts.
  • Capacity Multiplier: (3 + 1) / 3 = 1.33x (33% storage overhead).
  • A 500 GB VMDK consumes 665 GB of raw storage, saving 335 GB (33%) compared to RAID-1.
• vSAN ESA Adaptive RAID-5 (Exam-Critical Feature!): vSAN ESA dynamically adjusts its RAID-5 stripe width based on cluster size:
  • Small Clusters (3 to 5 hosts): ESA automatically implements a 2+1 stripe (2 data components + 1 parity component). This allows small 3-node and 4-node clusters to utilize RAID-5 erasure coding! The capacity multiplier is 1.5x (50% overhead). A 500 GB VMDK consumes 750 GB.
  • Large Clusters (6 or more hosts): ESA automatically adjusts to a 4+1 stripe (4 data components + 1 parity component). The capacity multiplier drops to 1.25x (only 25% overhead!). A 500 GB VMDK consumes only 625 GB of raw storage!

3. RAID-6 Erasure Coding (OSA and ESA)

RAID-6 provides dual-parity protection to withstand two concurrent failures (FTT=2):

• Component Architecture: Utilizes a 4+2 stripe (4 data components + 2 parity components).
• Host Requirement: Strictly requires a minimum of 6 hosts in both OSA and ESA.
• Capacity Calculation: Capacity Multiplier = (4 + 2) / 4 = 6 / 4 = 1.5x (50% overhead)
• Comparative Space Savings: Compare protecting a 10 TB dataset under FTT=2:
  • RAID-1 (FTT=2, 3.0x multiplier) consumes 30 TB of raw storage.
  • RAID-6 (FTT=2, 1.5x multiplier) consumes only 15 TB of raw storage.
  • RAID-6 delivers a 50% storage capacity reduction compared to RAID-1 while offering identical dual-failure protection.

---

Quorum, Voting, and Component Topologies

To prevent split-brain scenarios and data corruption during network partitions, vSAN enforces strict Quorum Voting Rules.

┌────────────────────────────────────────────────────────────────────────┐
│                     vSAN QUORUM VOTING LOGIC                           │
├────────────────────────────────────────────────────────────────────────┤
│ OBJECT AVAILABILITY RULE:                                              │
│ An object remains accessible IF AND ONLY IF:                           │
│        Total Accessible Component Votes > 50% of Total Votes           │
├────────────────────────────────────────────────────────────────────────┤
│ EXAMPLE: FTT=1 RAID-1 Mirroring                                        │
│ • Data Replica 1 (Host 1):  1 Vote                                     │
│ • Data Replica 2 (Host 2):  1 Vote                                     │
│ • Witness Component (Host 3): 1 Vote                                   │
│ Total Votes = 3. Majority Quorum = 2 Votes (>50%).                     │
│ If Host 1 fails: Surviving Votes = 2/3 (66%) ──> OBJECT ACCESSIBLE     │
│ If Host 1 + Host 2 fail: Surviving Votes = 1/3 (33%) ──> OBJECT HALTS  │
└────────────────────────────────────────────────────────────────────────┘

The Anatomy of Votes and Witnesses

• Every data replica and witness component is assigned an integer vote count. The total vote count for an object is always an odd number.
• Witness Components: A witness component contains zero bytes of user payload data. It consists solely of metadata (UUID, epoch counters, and 1 vote). Its sole purpose is to serve as an impartial tie-breaker.
• Network Partition Scenario: If a cluster partitions into two segments (e.g., 3 hosts on Partition Alpha and 2 hosts on Partition Beta), Partition Alpha holds majority quorum (>50% of votes). Virtual machines on Partition Alpha continue executing I/O. Partition Beta holds minority votes (<50%); vSAN immediately halts I/O on Partition Beta to prevent data divergence.

---

Component Failure States & The 60-Minute Repair Timer

When a physical device or host becomes unreachable, vSAN distinguishes between two fundamentally different component health states: Degraded and Absent.

┌────────────────────────────────────────────────────────────────────────┐
│                     COMPONENT FAILURE STATE LIFECYCLE                  │
├──────────────────────────────────┬─────────────────────────────────────┤
│         DEGRADED STATE           │            ABSENT STATE             │
├──────────────────────────────────┼─────────────────────────────────────┤
│ • Permanent hardware failure     │ • Transient disconnection           │
│ • Dead SSD, SMART trip, read err │ • Host reboot, network flap, maint. │
│ • REACTION: Immediate Rebuild!   │ • REACTION: Starts 60-Minute Timer! │
│   (Instant rebuild onto spare)   │   (clomd repair delay timer)        │
└──────────────────────────────────┴─────────────────────────────────────┘

1. Degraded Component State

• Cause: A permanent, unrecoverable hardware failure. Examples include an NVMe drive reporting fatal SMART media wear errors, an SSD controller death, or uncorrectable sector read failures.
• Operational Reaction: vSAN immediately recognizes that the drive cannot recover. It bypasses any waiting delay and initiates an immediate rebuild of all affected components onto surviving healthy drives in the cluster, provided sufficient slack space exists.

2. Absent Component State

• Cause: A transient, potentially temporary disconnection. Examples include an ESXi host rebooting after software patching, a temporary network cable disconnect, or a host entering maintenance mode.
• The Operational Problem: If an ESXi host with 10 TB of data reboots for a 5-minute BIOS update, immediately initiating a full rebuild across the network would flood the 25 GbE fabric, saturate CPUs, and waste drive endurance—only for the host to return 5 minutes later with all data intact.
• The Solution: The 60-Minute clomd Timer: When a component becomes unreachable due to a transient event, vSAN marks it Absent and starts the Cluster Level Object Manager daemon (clomd) repair delay timer. The default duration is 60 minutes (vsan.clomd.repairdelay = 60).
  • Host Returns Within 60 Minutes: The timer is aborted. Surviving components maintained an in-memory/disk bitmap of blocks changed while the host was offline. vSAN performs an ultra-fast incremental resynchronization, synchronizing only the modified delta blocks.
  • Host Does Not Return Within 60 Minutes: When the 60-minute timer expires, vSAN concludes the host is permanently lost. It changes the component status from Absent to Degraded and initiates a full, parallel reconstruction of all missing replicas and parity stripes from surviving healthy components onto remaining cluster capacity.

# Query the active CLOM repair delay setting on an ESXi host
esxcli system settings advanced list -o /VSAN/ClomRepairDelay

# Temporarily adjust the repair delay (e.g., to 120 minutes during prolonged maintenance)
esxcli system settings advanced set -o /VSAN/ClomRepairDelay -i 120

---

Advanced SPBM Rules & Quality of Service (QoS)

Beyond FTT and erasure coding, SPBM exposes advanced parameters to fine-tune workload behavior:

1. Number of Disk Stripes Per Object (Stripe Width): Configurable from 1 to 12. In legacy OSA, increasing stripe width partitioned a VMDK across multiple capacity drives within a disk group to prevent queue contention. In vSAN ESA, the Log-Structured File System automatically distributes writes across all NVMe drives, making manual stripe width tuning unnecessary.
2. Object Space Reservation (OSR): Governs thin vs. thick disk provisioning:
   • 0% (Default): Thin Provisioning. Storage blocks are allocated only when written by the guest OS.
   • 100%: Thick Provisioning. Slices raw capacity quota from the cluster upfront. In ESA, setting OSR=100% reserves capacity quota from the Storage Pool without physically zeroing blocks, preserving NVMe flash write endurance.
3. Number of IOPS Limit for Object (QoS): Enforces strict Quality of Service throttling at the hypervisor storage scheduler layer. Useful in multi-tenant private clouds to prevent "noisy neighbor" workloads from consuming excessive IOPS. If a VMDK exceeds its configured IOPS limit (e.g., 2,000 IOPS), the VMkernel delays I/O submission queues, smoothing storage demand across the cluster.

---

SPBM Sizing & Design Decision Matrix

---

Exam Watch: Key Scenarios and Candidate Traps

[!IMPORTANT] The ESA Adaptive RAID-5 Host Minimum: In vSAN OSA, RAID-5 strictly required 4 hosts. A favorite 2V0-17.25 exam question presents a 3-node vSAN ESA cluster and asks if RAID-5 can be applied. The correct answer is YES! In ESA, clusters with 3 to 5 hosts run a 2+1 adaptive RAID-5 scheme (1.5x overhead). It shifts to 4+1 (1.25x overhead) once 6 hosts are present.

[!WARNING] Degraded vs. Absent Component Timer: If an exam question describes a drive failure due to uncorrectable read/write errors or SMART death, do not wait 60 minutes. A Degraded component rebuilds immediately. The 60-minute clomd timer applies only to Absent components caused by transient disconnections (host reboots, network link losses).

[!TIP] Calculating RAID-6 Savings: Memorize the multiplier: RAID-6 is 1.5x. If an exam scenario asks for the raw storage required to protect 4 TB of data under FTT=2 using erasure coding: 4 TB * 1.5 = 6 TB. If using RAID-1 FTT=2, it would be 4 TB * 3.0 = 12 TB.