8.4 Third-Party Automation with Terraform & Git Workflows

8.4 Third-Party Automation with Terraform & Git Workflows

CloudOps Blueprint Focus: While AWS CloudFormation provides native orchestration, modern enterprise environments heavily utilize third-party Infrastructure as Code (IaC) tools—predominantly HashiCorp Terraform. For the AWS Certified CloudOps Engineer – Associate (SOA-C03) exam, you must understand Terraform architecture on AWS, master remote state management and distributed state locking with Amazon S3 and DynamoDB, manage configuration drift, adopt existing cloud resources, and secure CI/CD deployment pipelines using IAM OpenID Connect (OIDC) federation.

Terraform Fundamentals in AWS CloudOps

HashiCorp Terraform enables cloud engineers to define infrastructure using HashiCorp Configuration Language (HCL)—a declarative language. Rather than executing procedural scripts, declarative IaC defines the target end state; Terraform builds an internal dependency graph and executes the precise AWS API calls required to reconcile live infrastructure against declared configurations.

An AWS Terraform implementation begins with provider configuration:

terraform {
  required_version = ">= 1.5.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
  default_tags {
    tags = {
      Environment = "Production"
      ManagedBy   = "Terraform"
    }
  }
}

The default_tags block automatically attaches standardized organizational tags across all provisioned resources, enforcing tagging governance across the estate.

---

Core Terraform Operational Lifecycle Commands

The standard Terraform lifecycle follows four sequential CLI commands:

Additional validation tools include terraform fmt -check (verifies canonical syntax and indentation in CI lint stages) and terraform validate (validates syntax and internal references without calling remote APIs).

---

Remote State Management & Distributed State Locking

Terraform records resource attributes in a state file (terraform.tfstate). In team environments, local state files are prohibited—they expose plain-text secrets and cause state divergence.

Amazon S3 and DynamoDB State Backend Architecture

The enterprise standard pairs Amazon S3 for remote storage with Amazon DynamoDB for distributed locking:

1. Amazon S3 Bucket Requirements:

   • Versioning: Retains complete revision history of state files, enabling immediate recovery if corruption occurs.
   • Encryption: Enforces SSE-KMS using a Customer Managed Key (CMK) to protect sensitive data stored in state.
   • Bucket Policy: Blocks public access, mandates TLS encryption (aws:SecureTransport), and restricts access to deployment IAM roles.

2. Amazon DynamoDB State Locking:

   • Configured with a primary partition key named LockID (String type).
   • During terraform plan or terraform apply, Terraform writes a lock item containing the transaction ID. If another runner attempts an execution concurrently, DynamoDB rejects the write, and Terraform halts with an Error acquiring the state lock message, preventing concurrent state corruption.
   • Upon completion, Terraform deletes the lock item.

terraform {
  backend "s3" {
    bucket         = "corp-terraform-state-111122223333"
    key            = "prod/vpc/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    kms_key_id     = "arn:aws:kms:us-east-1:111122223333:key/abc-123"
    dynamodb_table = "corp-terraform-state-locks"
  }
}

---

Managing State Drift & Resource Importation

Reconciling Configuration Drift

State Drift occurs when infrastructure is modified out-of-band directly through the AWS Console or CLI. The modern, safe workflow uses terraform plan -refresh-only (or terraform apply -refresh-only). This queries live AWS APIs, identifies differences against state, and updates the state file to reflect reality without modifying or reverting cloud resources.

Adopting Unmanaged Infrastructure with Terraform Import

When adopting existing AWS resources into Terraform management without recreation, engineers use declarative import blocks:

import {
  to = aws_security_group.app_sg
  id = "sg-0123456789abcdef0"
}

Running terraform plan -generate-config-out=generated.tf automatically generates HCL configuration for the imported resource, binding it into state.

---

GitOps CI/CD Pipelines & Validation Workflows

In GitOps architectures, infrastructure repositories are structured by blast radius (e.g., separating networking, databases, and compute into isolated directories and state files). Automated CI/CD pipelines enforce quality gates:

1. Pull Request Validation: Runs terraform fmt -check, static security scanners (tfsec, checkov), terraform validate, and terraform plan -out=tfplan, posting speculative diffs to PR comments for peer review.
2. Main Branch Merge & Deployment: Merging to the protected main branch triggers automated deployment, executing terraform apply -auto-approve against the reviewed plan.

---

Credential-Less CI/CD Authentication via IAM OIDC Federation

Storing static IAM user access keys in CI/CD secret stores creates credential leakage risks. The AWS standard uses IAM OpenID Connect (OIDC) Identity Federation:

1. An IAM OIDC Identity Provider is registered in AWS trusting the CI/CD platform (e.g., token.actions.githubusercontent.com).
2. An IAM role is created with a trust policy allowing sts:AssumeRoleWithWebIdentity, strictly conditioned on repository claims (token.actions.githubusercontent.com:sub matching repo:my-org/infra:ref:refs/heads/main).

The CI/CD runner exchanges its short-lived OIDC JSON Web Token (JWT) with AWS STS to obtain temporary security credentials valid only for the pipeline run, eliminating persistent access keys entirely.