7.4 Variable Groups, Secret Variables & Key Vault Linking

7.4 Variable Groups, Secret Variables & Key Vault Linking

In enterprise DevOps delivery pipelines, decoupling code from configuration and establishing strict release governance are paramount. Hardcoding database connection strings, API tokens, or cloud credentials in source repositories violates DevSecOps standards and exposes organizations to credential exfiltration.

Azure Pipelines addresses configuration management and environment protection through three tightly coupled primitives:

1. Variable Groups: Centralized configuration and secret stores defined in the Azure DevOps Library.
2. Azure Key Vault Integration: Native synchronization of cloud secrets without persisting plaintext credentials in Azure DevOps.
3. YAML Environments with Approvals and Checks: Logical deployment targets guarded by automated and manual pre-deployment gates that developers cannot bypass.

Mastering these features is critical for designing secure, auditable release pipelines on the AZ-400 exam.

---

1. Azure DevOps Variable Groups & Secret Masking Mechanics

A Variable Group is an enterprise configuration asset created under Pipelines → Library in Azure DevOps. Variable Groups allow teams to define sets of variables that can be shared across multiple build and release pipelines within a project.

Linking Variable Groups in YAML

To consume a variable group in a YAML pipeline, reference it in the root or stage-level variables: block. You can combine multiple variable groups with individual inline variables:

variables:
  - group: GlobalPlatformConfig      # Group containing shared URLs and settings
  - group: PaymentServiceSecrets     # Group containing sensitive tokens
  - name: buildConfiguration
    value: 'Release'

Secret Variables and Log Masking

Variables can be designated as sensitive by clicking the lock icon next to the value in the Azure DevOps portal. Marking a variable as secret activates two critical security mechanisms:

1. Storage Encryption: The value is encrypted at rest using Azure DevOps service keys and is never exposed in plain text in the portal UI.
2. Log Masking: When an agent executes a step, the agent runtime registers all secret values in an internal redaction table. Any stdout or stderr stream emitting a secret value is intercepted, and the secret string is replaced with ***.

The Script Environment Variable Trap

By design, Azure Pipelines does not automatically inject secret variables as operating system environment variables into script tasks (script, bash, powershell). This prevents external malicious scripts or open-source dependencies from dumping the process environment to exfiltrate credentials.

To consume a secret variable inside a script, you must explicitly map it using the env: block:

# Correct Pattern: Explicit mapping via env block
- script: |
    echo "Authenticating with external payment gateway..."
    python deploy.py --token "$PAYMENT_TOKEN"
  displayName: 'Invoke Deployment Script'
  env:
    PAYMENT_TOKEN: $(PaymentApiKey) # Secret variable from Variable Group

# Anti-Pattern / Security Trap: Inlining secret into script command text
# - script: python deploy.py --token $(PaymentApiKey)
# Risk: Some interpreters print command strings to debugging logs before agent masking engages.

---

2. Azure Key Vault Linked Variable Groups

Rather than manually entering and rotating secrets inside Azure DevOps, enterprise security standards dictate sourcing secrets directly from Azure Key Vault.

[Azure Key Vault] ◄──(ARM Service Connection / Workload Identity)── [Azure DevOps Library]
       ▲                                                                   │
  Secret Rotated                                                   Linked Variable Group
  in Cloud Vault                                                           │
       │                                                                   ▼
       └────────────────── Next Pipeline Run Fetches Fresh Secret ─────────┘

Configuration Steps

1. In Azure DevOps, navigate to Pipelines → Library → + Variable group.
2. Enable the toggle: Link secrets from an Azure key vault as variables.
3. Select an Azure subscription and an authorized ARM Service Connection (preferably configured with Workload Identity Federation / OIDC).
4. Select the target Key Vault name.
5. Click + Add to select individual secrets, or select all secrets.
6. Save the variable group.

Secret Synchronization and Caching Behavior

• Run Initialization Fetch: Azure Key Vault linked variable groups do not maintain a persistent cache of secret values in Azure DevOps. When a pipeline run is queued, the orchestration engine authenticates to Azure Key Vault via the service connection and downloads the current secret versions.
• Zero Pipeline Modification on Secret Rotation: When an administrator or automated process rotates a secret in Azure Key Vault, the subsequent execution of the Azure Pipeline automatically retrieves the new secret value without modifying the pipeline YAML or touching Azure DevOps settings.
• Variable Group vs. AzureKeyVault@2 Task:
  • Variable Group: Secrets are resolved at pipeline initialization and can be scoped cleanly across entire stages or pipelines.
  • AzureKeyVault@2 Task: A specific pipeline task that runs on an agent to retrieve secrets dynamically during job execution. Used when secrets must be queried conditionally or dynamically at runtime using runtime parameters.

---

3. Variable Scoping, Precedence and Group Security

Variables can be declared in several places, and the exam tests which one wins. For YAML pipelines Microsoft documents this order, highest precedence first:

The counter-intuitive result is the one the exam tests: in a YAML pipeline a value hard-coded in the YAML file beats a queue-time override, which is the opposite of the classic release-pipeline model where queue time is strongest. If a variable must be overridable at queue time, do not also define it in the YAML file.

A variable group contributes its variables at the scope where it is linked, so a group linked at stage scope exists only in that stage. Within a single scope the last definition wins, so a variables: block that lists a group and then a literal name/value pair overrides the group's value for that name.

Two syntax families behave completely differently and cause most misconfigurations:

• $(var) is a runtime macro: it is substituted when the task executes, so it can carry values produced earlier in the run, and it silently leaves the literal text in place if the variable does not exist.
• ${{ variables.var }} is a compile-time template expression: it is resolved while the YAML is being parsed, before any agent is assigned, so it can drive condition, dependsOn and template selection but can never see a value produced during the run.

Variable-group security is a separate plane from pipeline authoring. Each group carries Reader / User / Administrator roles plus a pipeline permissions list. A group with "Open access" is usable by every pipeline in the project, which is exactly how a low-trust sandbox pipeline gains production credentials. Restrict production groups to the named release pipelines instead.

# Create a group and restrict it to a single pipeline
az pipelines variable-group create --name prod-release --authorize false   --variables REGION=eastus2 --organization "$ORG" --project "$PROJECT"

Because secret variables are not injected into the script environment automatically, map them explicitly with an env: block; otherwise $(SqlPassword) inside an inline script expands to an empty string.