9.4 Virtual Machine Scale Sets, Autoscale & Azure Dedicated Hosts

Virtual Machine Scale Sets, Autoscale & Azure Dedicated Hosts

1. Virtual Machine Scale Sets (VMSS): Flexible vs Uniform Orchestration

Azure Virtual Machine Scale Sets (VMSS) allow administrators to deploy and manage a group of load-balanced virtual machines. VMSS supports two distinct orchestration modes:

+-----------------------------------------------------------------------------+
|                 VMSS UNIFORM VS FLEXIBLE ORCHESTRATION MODES                |
|                                                                             |
|   UNIFORM ORCHESTRATION MODE            FLEXIBLE ORCHESTRATION MODE         |
|   - Identical VMs from single model     - Heterogeneous VM sizes & OS types |
|   - Golden image deployment             - Mix Spot & Pay-as-You-Go VMs      |
|   - Scalability: Up to 1,000 instances  - Individual VM lifecycle management|
|   - Hidden individual VM objects        - Standard VM APIs & ARM templates  |
|   - Best for: Cloud-Native Stateless    - Best for: Statefully Managed App  |
|     Web Frontends / Worker Farms          Stacks, Hybrid Windows Workloads  |
+-----------------------------------------------------------------------------+

Detailed Orchestration Comparison:

1. Uniform Orchestration Mode (Legacy / Cloud-Native):

   • Uses a centralized VM template model. Every instance in the scale set is identical in size, image, and disk configuration.
   • Supports scaling up to 1,000 instances with platform images (or 600 with custom images).
   • Individual VM instances are child resources of the scale set and cannot be managed independently using standard Azure VM management commands.

2. Flexible Orchestration Mode (Modern Enterprise Standard):

   • Provides unified management across identical or heterogeneous virtual machines.
   • Heterogeneous Sizing: Mix different VM series, sizes, and OS versions within the same scale set.
   • Cost Optimization: Mix Spot VMs (for fault-tolerant batch computing at deep discounts) with Standard Pay-as-You-Go VMs in defined ratios within a single scale set.
   • High Availability Spreading: Automatically spreads instances across Fault Domains (up to 5 FDs) within a region or across Availability Zones (Zone 1, 2, 3).
   • Full VM Lifecycle Support: Every VM is a standard first-class Azure VM object. Supports standard Azure Backup, Azure Site Recovery, individual disk attachments, and Azure Arc management.

---

2. Autoscale Policies: Metric-Based vs Schedule-Based Scaling

Azure Monitor Autoscale dynamically increases (scale out) or decreases (scale in) the number of VMSS instances to match application demand.

+-----------------------------------------------------------------------------+
|                       AUTOSCALE DECISION ENGINE & FLAPPING                  |
|                                                                             |
|   [METRIC COLLECTION] ---> Azure Monitor Agent collects CPU / Memory / Queue|
|                                     |                                       |
|                                     v                                       |
|   [RULE EVALUATION]   ---> Is Average CPU > 75% for 10 consecutive minutes? |
|                                     |                                       |
|                                     v (YES)                                 |
|   [SCALE-OUT ACTION]  ---> Increase instance count by +2 VMs                |
|                                     |                                       |
|                                     v                                       |
|   [COOL-DOWN TIMER]   ---> Block all scaling actions for 20 minutes         |
|                            (Prevents Rapid Oscillation / 'Flapping')        |
+-----------------------------------------------------------------------------+

Autoscale Rule Architecture:

1. Metric-Based Scaling Rules:

   • Host Compute Metrics: Default platform metrics like Percentage CPU or Network In/Out collected without guest agents.
   • Guest OS Metrics: Memory demand (e.g., Available Memory Bytes), disk queue depth, or custom Windows Performance Counters collected via the Azure Monitor Agent (AMA).
   • Application Queue Metrics: Azure Service Bus queue length or Azure Storage Queue message counts (e.g., scale out when queue depth exceeds 500 messages per instance).

2. Scale-Out vs. Scale-In Asymmetry:

   • Scale-Out Rule: Aggressive threshold (e.g., scale out by 2 instances when Average CPU > 75% over 5 minutes).
   • Scale-In Rule: Conservative threshold (e.g., scale in by 1 instance when Average CPU < 25% over 15 minutes).
   • Why? Prevents premature deallocation of instances during minor traffic lulls.

3. Cool-Down Periods & Flapping Prevention:

   • The cool-down period is a mandatory waiting time (typically 10 to 30 minutes) after a scaling action during which the autoscale engine ignores metric triggers.
   • Flapping: A disastrous condition where a scale-in action reduces capacity, immediately causing remaining VMs to breach high-utilization thresholds, triggering an immediate scale-out action. Cool-down periods allow new VMs to boot, warm up caches, and stabilize telemetry before further scale actions occur.

4. Schedule-Based Scaling Rules:

   • Overrides default metric profiles during predictable calendar events.
   • Example: Scale the base instance count from 2 to 10 instances every Monday through Friday from 08:00 to 18:00 UTC, returning to 2 instances during weekends and off-peak hours.

---

3. Rolling Upgrades & Automatic OS Image Upgrades for Windows VMSS

Maintaining guest OS patch compliance across hundreds of Windows Server VMSS instances without application downtime requires Rolling Upgrade Policies.

+-----------------------------------------------------------------------------+
|                   VMSS ROLLING UPGRADE BATCH EXECUTION                      |
|                                                                             |
|   [BATCH 1: 20% Instances] ---> Upgrade OS -> Wait for Health Probe OK ----+||
|                                                                            ||
|                                                                            v|
|   [BATCH 2: 20% Instances] ---> Upgrade OS -> Wait for Health Probe OK ----+||
|                                                                            ||
|                                                                            v|
|   [BATCH 3: 20% Instances] ---> Upgrade OS -> Wait for Health Probe OK ----+||
|                                                                            ||
|   (If Health Probe fails at any batch -> ROLLBACK & HALT UPGRADE)          v|
+-----------------------------------------------------------------------------+

• Upgrade Modes:
  • Manual: Instances are updated only when explicitly initiated by an administrator via PowerShell or CLI.
  • Automatic: When a new VMSS model or image is committed, Azure automatically upgrades all instances simultaneously (causes downtime unless orchestrated).
  • Rolling: Azure updates instances in controlled batches (e.g., 20% max batch size).
• Application Health Extension:
  • Deployed inside the Windows Server guest OS to monitor local application health (e.g., testing http://localhost:80/health).
  • Azure verifies the Application Health Extension reports Healthy before moving to the next rolling upgrade batch.
• Automatic OS Image Upgrades:
  • When Microsoft publishes monthly Windows Server security patches to the Azure Marketplace, the scale set automatically rolls out the updated base image batch by batch with zero downtime.

---

4. Azure Dedicated Hosts

An Azure Dedicated Host provides a physical hardware server in an Azure datacenter that is completely dedicated to running a single customer's virtual machines.

+-----------------------------------------------------------------------------+
|                      AZURE DEDICATED HOST ARCHITECTURE                      |
|                                                                             |
|   [DEDICATED HOST GROUP (Region: East US, Zone 1, FD: 2)]                   |
|   +---------------------------------------------------------------------+   |
|   |  Physical Server (Dedicated Single Tenant)                          |   |
|   |  - Intel Xeon Hardware Isolation (Complies with HIPAA / DoD / PCI)  |   |
|   |  - Maintenance Control Window (Defer OS hypervisor patching 35 days)|   |
|   |  - Azure Hybrid Benefit (License physical cores for unlimited VMs)  |   |
|   |                                                                     |   |
|   |  +---------------------------------------------------------------+  |   |
|   |  | Guest VMs: [VM-DC01] [VM-SQL01] [VM-APP01] [VM-FINANCE01]     |  |   |
|   |  +---------------------------------------------------------------+  |   |
|   +---------------------------------------------------------------------+   |
+-----------------------------------------------------------------------------+

Key Enterprise Benefits of Dedicated Hosts:

1. Hardware Isolation: Guarantees no other customer's VMs run on the physical server, satisfying strict regulatory frameworks (PCI-DSS, HIPAA, FedRAMP High).
2. Maintenance Control: Allows administrators to control and defer platform maintenance updates for up to 35 days, scheduling host patching during planned maintenance windows.
3. Licensing Optimization with Azure Hybrid Benefit: By licensing all physical processor cores on the Dedicated Host with Windows Server Datacenter edition (with Software Assurance), customers obtain unlimited Windows Server virtualization rights on that host without licensing individual VMs.

PowerShell Deployment: VMSS Flexible Mode with Autoscale

# Create a Resource Group and Virtual Network
$rg = 'RG-ScaleSets'
$loc = 'eastus'
New-AzResourceGroup -Name $rg -Location $loc

# Create a VMSS in Flexible Orchestration Mode across 3 Availability Zones
$vmssConfig = New-AzVmssConfig `
    -Location $loc `
    -OrchestrationMode 'Flexible' `
    -PlatformFaultDomainCount 1 `
    -Zone '1','2','3'

Set-AzVmssStorageProfile $vmssConfig `
    -OsDiskCreateOption 'FromImage' `
    -OsDiskCaching 'ReadWrite' `
    -ImageReferencePublisher 'MicrosoftWindowsServer' `
    -ImageReferenceOffer 'WindowsServer' `
    -ImageReferenceSku '2022-datacenter-azure-edition' `
    -ImageReferenceVersion 'latest'

# Commit VMSS deployment
New-AzVmss -ResourceGroupName $rg -VMScaleSetName 'VMSS-AppFlex' -VirtualMachineScaleSet $vmssConfig