12.3 Artifact & Build Retention Governance

12.3 Artifact & Build Retention Governance

Optimization decides how fast a pipeline runs; retention decides how long its output survives. Retention policy is where storage cost meets audit obligation, and Azure DevOps separates the run record, the artifacts, the test results and the symbols so each can be kept for a different period.

1. Artifact and Build Retention Governance

Continuous delivery produces massive quantities of pipeline execution logs, published drop artifacts, symbols, and test attachments. Without automated retention governance, storage consumption escalates costs and exposes organizations to compliance risks.

The Modern Project-Level Retention Model

In modern Azure DevOps, retention policies are managed at the Project Settings level rather than configured individually on each pipeline:

• Navigate to Project Settings → Pipelines → Settings.
• Days to keep runs: Specifies how many days pipeline run history, logs, and test results are retained (Default: 30 days; Range: 1 to 730 days).
• Minimum runs to keep per pipeline: Guarantees that at least $N$ successful runs are preserved regardless of age, ensuring low-frequency pipelines do not lose their baseline history.
• Days to keep pull request runs: Typically configured for a shorter period (e.g., 7 to 10 days) since PR branches are short-lived.

Artifacts vs. Test Results vs. Symbols Retention

┌─────────────────────────────────────────────────────────────────────────────────┐
│                     RETENTION CATEGORIES IN AZURE PIPELINES                     │
├──────────────────────┬──────────────────────────────────────────────────────────┤
│ Data Category        │ Retention Mechanics & Governance                         │
├──────────────────────┼──────────────────────────────────────────────────────────┤
│ Pipeline Artifacts   │ Dropped zips/binaries published via PublishPipelineArtifact│
│ (Build Drops)        │ Deleted when the pipeline run expires. Can set specific  │
│                      │ retention per artifact or inherit run lifecycle.         │
├──────────────────────┼──────────────────────────────────────────────────────────┤
│ Test Results & TRX   │ Stored in Azure Test Plans analytics data store.         │
│ Files                │ Governed by project retention settings; aggregated data  │
│                      │ preserved for long-term reporting.                       │
├──────────────────────┼──────────────────────────────────────────────────────────┤
│ Symbol Files (PDBs)  │ Published via IndexSourcesAndPublishSymbols@2 to Azure   │
│                      │ DevOps Symbol Server. Configured with an independent     │
│                      │ expiration window to support post-mortem debugging.      │
├──────────────────────┼──────────────────────────────────────────────────────────┤
│ Azure Artifacts      │ NuGet, npm, Maven feeds governed by Feed Retention       │
│ (Package Feeds)      │ Policies: maximum versions per package (e.g., retain 10).│
└──────────────────────┴──────────────────────────────────────────────────────────┘

Pipeline Retention Leases

What happens when a build is deployed to Production and must be retained for 7 years to satisfy regulatory audits (e.g., SOX, HIPAA, or PCI-DSS)?

[Pipeline Run #4128 Executes]
              │
              ▼
[Deploys to Environment: 'Production']
              │
              ▼
[Azure Pipelines Creates Automated RETENTION LEASE]
              │
    ┌─────────┴─────────┐
    ▼                   ▼
[30-Day Project    [Retention Lease ACTIVE]
 Retention Policy]  • Prevents automated deletion
 Attempts Cleanup   • Retains build run, logs, and drop artifacts
                    • Preserved until lease is explicitly removed

• Automated Retention Leases on Environments: When a pipeline executes a deployment job targeting an Azure DevOps Environment (e.g., environment: 'Production'), Azure Pipelines automatically attaches a retention lease to that specific pipeline run.
• Lease Protection: A pipeline run protected by an active retention lease will never be deleted by the automated retention cleanup, even if it exceeds the 30-day project retention setting.
• Manual & REST API Leases: Compliance officers or automation scripts can programmatically create retention leases via the Azure DevOps REST API:

  POST https://dev.azure.com/{organization}/{project}/_apis/build/retention/leases?api-version=7.1-preview.1
  Content-Type: application/json
  
  [
    {
      "buildId": 4128,
      "daysValid": 2555,
      "definitionId": 12,
      "ownerId": "ComplianceAuditOfficer",
      "protectPipeline": true
    }
  ]
  

---

2. Comprehensive Optimization Techniques Comparison Table

---

3. Realistic Exam Scenario & Common Traps

Scenario: Global Healthcare SaaS Compliance & Speed Optimization

Context: MedTech Cloud builds a HIPAA-compliant healthcare portal. The multi-stage pipeline currently takes 55 minutes to execute:

1. Dependency installation (npm ci and dotnet restore) takes 12 minutes on every run.
2. The 3,500 integration tests take 35 minutes running sequentially on a single Linux agent.
3. HIPAA regulations mandate that every build artifact and test execution log deployed to the Production-HIPAA environment must be retained for exactly 7 years (2,555 days).
4. Routine feature branch validation builds must be deleted after 14 days to prevent storage cost escalation.

DevOps Solution:

• Optimization: Implement the Cache@2 task for both npm (package-lock.json key) and NuGet (packages.lock.json key), reducing dependency resolution from 12 minutes to 45 seconds.
• Parallelization: Configure strategy: parallel: 5 on the test job and configure VSTest@2 test slicing, reducing the 35-minute test phase to approximately 7 minutes.
• Retention Governance: Configure Project Settings pipeline retention to 14 days for standard runs. Configure the deployment stage using a deployment: job targeting the Production-HIPAA environment. Azure Pipelines automatically creates an environment retention lease, and an automated REST call or environment retention policy sets the lease duration to 2,555 days.

Common Exam Traps to Avoid

• Trap: Caching Dynamic or Volatile Output Folders: Caching folders containing build timestamps or volatile binaries without hashing their inputs results in stale, unrepeatable builds.
• Trap: Believing Cache@2 Saves on Failure: The cache upload occurs in a post-job step that runs only if the job succeeds. If tests fail, the cache is not saved, ensuring that corrupted states are not cached.
• Trap: Confusing Project Retention with Azure Artifacts Feed Retention: Project retention settings govern pipeline runs, logs, and pipeline artifacts. Package feeds in Azure Artifacts (NuGet/npm feeds) have their own independent Feed Retention Policies governing package versions.