6.4 Azure Automation for Hybrid Workloads: Runbooks & Hybrid Runbook Workers

Azure Automation for Hybrid Workloads: Runbooks & Hybrid Runbook Workers

Azure Update Manager solves patching. Everything else operational — restarting a stuck service on a branch file server at 03:00, exporting an Active Directory system state before a change window, reclaiming space on a Hyper-V host — is a runbook problem. Runbooks live in an Azure Automation account, but by default they execute in an Azure-hosted sandbox that cannot see a single on-premises resource. The Hybrid Runbook Worker closes that gap by moving execution onto a machine inside your network while leaving authoring, scheduling and credential storage in Azure.

---

1. The Hybrid Runbook Worker Model

While Azure Update Manager handles operating system patching, complex operational automation—such as stopping database services prior to patching, restarting clustered Hyper-V nodes sequentially, or managing local Active Directory accounts—requires Azure Automation.

The Extension-Based Hybrid Runbook Worker (HRW) Model

A Hybrid Runbook Worker (HRW) executes PowerShell and Python runbooks hosted in an Azure Automation Account directly within the on-premises operating system.

+-----------------------------------------------------------------------------------------+
|                 EXTENSION-BASED HYBRID RUNBOOK WORKER ARCHITECTURE                      |
|                                                                                         |
|   [AZURE AUTOMATION ACCOUNT]                                                            |
|   - PowerShell Runbooks (e.g., Invoke-ClusterPatching.ps1)                              |
|   - Shared Assets: Credentials, Variables, Certificates                                 |
|                                 |                                                       |
|                                 | (Job Dispatched via ARM)                              |
|                                 v                                                       |
|   [ON-PREMISES ARC-ENABLED SERVER]                                                      |
|   +---------------------------------------------------------------------------------+   |
|   | Azure Connected Machine Agent                                                   |   |
|   |  - Extension Manager (extmd) installs:                                          |   |
|   |    'AzureAutomationWindowsWorkforceExtension' (Hybrid Worker VM Extension)      |   |
|   |                                                                                 |   |
|   | In-Guest Execution Engine:                                                      |   |
|   |  - Executes Runbook locally under NT AUTHORITY\SYSTEM context                   |   |
|   |  - Authenticates to Azure via Server's System-Assigned Managed Identity         |   |
|   |  - Interacts with Local Active Directory, Hyper-V, IIS, and Local Storage       |   |
|   +---------------------------------------------------------------------------------+   |
+-----------------------------------------------------------------------------------------+

Extension-Based HRW vs Legacy Agent-Based HRW:

• Legacy Agent-Based HRW: Depended on the legacy Microsoft Monitoring Agent (MMA) and required configuring complex Azure Automation "Run As Accounts" (Service Principals with certificates expiring annually).
• Extension-Based HRW (Current Standard): Deployed as a native VM extension (AzureAutomationWindowsWorkforceExtension) on Arc-enabled servers. It utilizes the machine's System-Assigned Managed Identity for authentication to Azure resources, eliminating certificate rotation and credential management.

Authoring & Executing Runbooks with Shared Assets

<#
    .SYNOPSIS
        PowerShell Runbook executed on an on-premises Hybrid Runbook Worker.
        Backs up local Active Directory and purges temporary maintenance files.
#>

# Retrieve shared automation variable and credentials from Azure Automation Account
$BackupPath = Get-AutomationVariable -Name 'LocalADBackupRootPath'
$AdminCred  = Get-AutomationPSCredential -Name 'OnPremServiceAccountCred'

Write-Output "Starting local maintenance runbook on worker: $env:COMPUTERNAME"

# Perform local operating system and workload tasks
if (Test-Path -Path $BackupPath) {
    Write-Output "Exporting AD System State to $BackupPath..."
    # Execute local PowerShell commands against on-premises subsystems
    Get-Service -Name NTDS | Restart-Service -Force
    Write-Output "NTDS Service verified successfully."
} else {
    Write-Error "Target backup directory $BackupPath does not exist locally!"
}

[!TIP] Hybrid Worker Groups for High Availability: In production environments, always deploy multiple Arc-enabled servers into a Hybrid Worker Group. When an Azure Automation job is dispatched to a Hybrid Worker Group, Azure automatically distributes runbook execution to an available, healthy worker node, ensuring operational continuity if an individual on-premises server is rebooting during maintenance.

---

2. The Retirement That Changes the Correct Answer

There were two worker platforms, and only one survives:

For an on-premises server this means the sequence is always: onboard to Azure Arc first so the Connected Machine agent is present, then install the Hybrid Runbook Worker extension through that agent. Any design that starts by installing the Log Analytics agent is describing a retired product.

Extension-based workers also inherit automatic minor-version upgrades by default, and can be deployed through the portal, PowerShell, Azure CLI, ARM or Bicep templates, REST, or the Extensions blade of an individual machine.

3. Worker Groups, Scale and Job Dispatch Behaviour

A Hybrid Runbook Worker group is the unit you target when you start a runbook — you choose the group, never an individual worker. Adding a second worker to a group buys both high availability and load balancing.

Two consequences follow directly. First, if you are pushing jobs faster than roughly four per 30 seconds into a single-worker group, jobs will queue and eventually error — the fix is more workers in the group, not a bigger worker. Second, Hybrid Runbook Workers are not supported on Virtual Machine Scale Sets, so the "scale out the workers automatically" design does not exist.

The other genuine advantage is the absence of sandbox limits: a Hybrid Runbook Worker job is bounded only by the host's own CPU, memory, disk and network, and is not subject to the Azure sandbox fair-share three-hour ceiling. Long-running migrations and bulk operations are the canonical reason to choose a worker over a cloud job.

System versus User Hybrid Workers

A system Hybrid Worker group is created and managed by Azure services such as Update Management and Change Tracking on your behalf and cannot be used to run your own runbooks. A user Hybrid Worker group is the one you create and target explicitly. Selecting a system group in a runbook schedule is a configuration error, not a shortcut.

4. Execution Context, Identity and Networking

On Windows, runbook jobs on a Hybrid Runbook Worker execute under the local SYSTEM account (on Linux, under nxautomation). That is convenient — local file, service and registry operations need no stored credential — and it is also a limitation, because SYSTEM has no network identity of its own beyond the computer account. Anything that must authenticate off-box should use either the machine's system-assigned managed identity for Azure resources, or a credential retrieved from the Automation account's shared assets for on-premises resources:

# Azure resources: use the machine's managed identity, no secrets stored anywhere
Connect-AzAccount -Identity
Get-AzVM -ResourceGroupName 'rg-branch-infra' | Select-Object Name, PowerState

# On-premises resources: pull a credential from the Automation account's shared assets
$SvcCred = Get-AutomationPSCredential -Name 'OnPremServiceAccountCred'
Invoke-Command -ComputerName 'FS-BRANCH-07' -Credential $SvcCred -ScriptBlock { Restart-Service Spooler }

For network controls, Azure Automation publishes the GuestAndHybridManagement service tag, which can be used in a network security group or Azure Firewall rule in place of hard-coded IP addresses. It covers webhook triggers from inside a virtual network and worker-to-service communication — but note that it does not cover runbook execution inside an Azure sandbox, only execution on a Hybrid Runbook Worker.

Shared assets are what keep runbooks portable: variables (optionally encrypted), credentials, certificates, connections and schedules are stored in the Automation account and retrieved at run time with Get-AutomationVariable, Get-AutomationPSCredential and Get-AutomationCertificate. Storing a password in the runbook body instead is the failure the exam expects you to reject.