15.3 Audit Evidence Collection & Well-Architected Security Reviews

15.3 Audit Evidence Collection & Well-Architected Security Reviews

Enterprise security teams operating in regulated industries—including banking, healthcare, government, and payment processing—face continuous audit scrutiny from internal governance bodies and external regulatory examiners. Demonstrating compliance requires proving that security policies, operational procedures, and technical guardrails are continuously operating as intended. Historically, preparing for compliance audits required labor-intensive, point-in-time evidence collection: taking hundreds of console screenshots, exporting log files, parsing spreadsheets, and interviewing engineering leads.

AWS provides three specialized services to eliminate manual audit friction and formalize architectural governance: AWS Audit Manager, AWS Artifact, and the AWS Well-Architected Tool. Understanding the distinct scope, evidence collection mechanics, and shared responsibility boundaries of each service is essential for cloud security architects and a core focus of the Specialty certification.

---

AWS Audit Manager Architecture & Automated Evidence Collection

AWS Audit Manager automates evidence gathering to assess whether cloud usage adheres to industry standards, legal regulations, and internal corporate controls. Audit Manager continuously monitors AWS accounts and automatically collects, organizes, and cryptographically timestamps audit evidence across distributed environments.

Audit Manager Evidence Pipeline:
┌──────────────────────────────────────────┐
│ Automated Data Sources:                  │
│ 1. AWS CloudTrail (API & User Activity)  │
│ 2. AWS Config (Resource Configurations)  │ ──> [Audit Manager Assessment] ──> [Auditor-Ready Report]
│ 3. AWS Security Hub (Security Findings)  │          │                               (PDF + Cryptographic
│ 4. AWS Checkers (Direct API Evaluations) │          ├── Standard / Custom Frameworks         Manifest)
└──────────────────────────────────────────┘          ├── Control Sets & Controls
                     ▲                                └── Manual Evidence Uploads
                     │
            [Organization Scope]

Core Audit Manager Concepts

• Frameworks: Pre-packaged collections of control sets mapped to specific regulatory baselines or internal policies. Audit Manager provides pre-built Standard Frameworks (e.g., SOC 2 Type II, PCI DSS v4.0, HIPAA Security Rule, ISO/IEC 27001, FedRAMP Moderate, GDPR) and allows organizations to build Custom Frameworks by combining custom controls with standard controls.
• Controls: Individual compliance requirements within a framework (e.g., "Enforce multi-factor authentication for administrative users" or "Ensure S3 buckets enforce encryption in transit").
• Assessments: An active audit project created within Audit Manager. When an assessment is created, the user selects the target framework, defines the scope (specific AWS accounts in an AWS Organization), and specifies designated audit team members.
• Evidence Folders: Structured containers where Audit Manager continuously deposits collected evidence, organized hierarchically by control set and control.

The Four Automated Evidence Sources

Audit Manager continuously harvests evidence from four distinct automated sources:

1. AWS CloudTrail: Telemetry evidence capturing user activity, administrative actions, and API events. For example, CloudTrail evidence proves who accessed a resource, modified a security group, or executed a privileged role assumption.
2. AWS Config: Resource configuration compliance states evaluated by AWS Config rules. Config evidence records whether a resource property matched the desired compliance definition at a given point in time.
3. AWS Security Hub: Security posture checks and finding evaluations across enabled standards (e.g., AWS Foundational Security Best Practices, CIS Benchmarks). This evidence demonstrates overall control adherence across integrated services.
4. AWS API Calls (Service Checkers): Direct service state checks executed natively by Audit Manager checkers. Checkers query service APIs directly without requiring Config rules (e.g., querying IAM password policy parameters or checking KMS key rotation status).

Manual Evidence Uploads

Not all audit requirements can be validated through automated cloud telemetry. Regulatory standards frequently require proof of administrative and human processes—such as employee security awareness training records, business continuity plan documents, third-party vendor penetration testing reports, or data retention policy sign-offs. Audit Manager supports Manual Evidence Uploads, allowing security leads to upload PDF, image, or text documents directly into specific controls alongside automated evidence.

Assessment Reports & Cryptographic Integrity

Once an assessment has collected sufficient evidence over an audit window (e.g., a 6-month or 12-month SOC 2 evaluation period), the security team generates an Assessment Report:

• Audit Manager compiles selected evidence folders into an auditor-ready package consisting of an executive PDF summary and detailed evidence files.
• The package includes a cryptographic manifest file containing SHA-256 hashes of all evidence items. This cryptographic hash proves to external auditors that evidence files have not been modified, tampered with, or altered since their automated capture.
• Evidence can't be edited in Audit Manager and is retained for two years from creation; export reports you must keep longer.

AWS Artifact: Compliance Reports & Legal Agreements

While AWS Audit Manager collects evidence regarding the customer's cloud configurations, AWS Artifact is the dedicated self-service portal for accessing compliance documentation and legal agreements concerning AWS's own infrastructure.

Under the AWS Shared Responsibility Model, customers cannot physically audit AWS data centers or inspect physical host hypervisors. Instead, customers rely on independent third-party auditor attestations published by AWS in AWS Artifact.

AWS Artifact Portal:
├── AWS Artifact Reports (Third-Party Attestations OF the Cloud)
│   ├── SOC 1 (SSAE 18 / ISAE 3402) Type II Reports
│   ├── SOC 2 Type II Security, Availability & Confidentiality Reports
│   ├── SOC 3 General Use Public Reports
│   ├── ISO Certifications (ISO 27001, 27017, 27018, 9001, 22301)
│   ├── FedRAMP Package (SSP, SAP, SAR, and Authorization to Operate)
│   └── PCI DSS Attestation of Compliance (AOC) & Responsibility Matrix
│
└── AWS Artifact Agreements (Legal Contracts between Customer & AWS)
    ├── Business Associate Addendum (BAA) for HIPAA / HITECH Compliance
    ├── Non-Disclosure Agreements (NDA)
    └── Federal Financial Institutions Examination Council (FFIEC) Materials

AWS Artifact Reports

• Service Organization Control (SOC) Reports:
  • SOC 1 Type II: Evaluates controls relevant to internal control over financial reporting.
  • SOC 2 Type II: Comprehensive technical audit evaluating AWS infrastructure security, availability, processing integrity, and confidentiality over a multi-month period.
  • SOC 3: Summary version of the SOC 2 report intended for general public distribution without requiring an NDA.
• Payment Card Industry Data Security Standard (PCI DSS): Provides the Attestation of Compliance (AOC) and the PCI Responsibility Matrix, which outlines exactly which PCI DSS requirements are handled by AWS and which must be configured by the customer.
• Confidentiality & Watermarking: Many reports in AWS Artifact are confidential proprietary documents. When an authorized IAM principal downloads a report, AWS automatically embeds a personalized, cryptographic watermark containing the downloader's account details and generates an NDA agreement acceptance record.

AWS Artifact Agreements & HIPAA BAA

Organizations handling regulated workloads must execute legal agreements with AWS before ingesting protected data:

• Business Associate Addendum (BAA): Mandatory contract under the Health Insurance Portability and Accountability Act (HIPAA) required before storing or processing Protected Health Information (PHI) on AWS.
• Organization Agreements: When executed by the management account in AWS Organizations, an agreement (such as the BAA) can be accepted on behalf of all current and future accounts within the AWS Organization, eliminating the need to execute separate agreements per account.

---

AWS Well-Architected Tool & Security Pillar Reviews

The AWS Well-Architected Framework Tool provides a formal, structured mechanism for measuring cloud architectures against AWS architectural best practices across the six Well-Architected pillars: Operational Excellence, Security, Reliability, Performance Efficiency, Cost Optimization, and Sustainability.

The 7 Security Pillar Design Principles

When conducting a Security Pillar review, the Well-Architected Tool evaluates workloads against seven core foundational principles:

1. Implement a strong identity foundation: Enforce least privilege, centralize identity management, and eliminate reliance on static long-term credentials.
2. Enable traceability: Log, monitor, and alert on all API activities and metrics in real time; integrate log telemetry into automated response pipelines.
3. Apply security at all layers: Implement defense-in-depth across edge networks, VPC perimeters, compute instances, operating systems, and application code.
4. Automate security best practices: Automate software-defined guardrails, IaC security gating, and incident response runbooks to reduce manual operational risk.
5. Protect data in transit and at rest: Enforce cryptographic protection across all storage tiers, communication channels, and secret management stores.
6. Keep people away from data: Eliminate direct human access to production databases and storage systems; mandate processing via automated tools, APIs, and CI/CD pipelines.
7. Prepare for security events: Establish incident response plans, run automated game days, and develop automated containment and forensic investigation playbooks.

Well-Architected Review Mechanics

• Workload Definition: Users define a workload by specifying its business purpose, environment type (Production vs Non-Production), and operational AWS accounts.
• Lenses: In addition to the standard Well-Architected Base Lens, organizations can apply specialized lenses (e.g., Serverless Lens, Financial Services Lens, Machine Learning Lens, or Custom Lenses authored by enterprise security teams).
• Risk Identification:
  • High Risk Issues (HRIs): Critical architectural deficiencies that represent significant security vulnerabilities, single points of failure, or severe operational risks.
  • Medium Risk Issues (MRIs): Deviations from best practices that present moderate risk to workload reliability or governance.
• Milestones: Formal point-in-time snapshots created to document the state of a workload before and after remediation initiatives.
• Improvement Plans: The tool automatically generates a prioritized, step-by-step remediation guide listing exact architectural recommendations to eliminate detected HRIs and MRIs.
• Workload Sharing: Workloads can be shared across AWS accounts or across an entire AWS Organization using AWS Resource Access Manager (RAM) or direct account sharing.

---

Compliance and Auditing Service Comparison Matrix

Understanding which service to select based on specific auditing and governance requirements is critical for the Specialty exam:

---

Specialty Exam Pitfalls & Architectural Traps

1. The Artifact vs. Audit Manager Scope Confusion: Confusing where to obtain proof of data center physical security versus proof of customer bucket encryption. External auditors requesting proof of AWS physical data center controls or hypervisor isolation must be directed to AWS Artifact for the AWS SOC 2 Type II report. If auditors request proof that the customer's production S3 buckets are encrypted, the team must use AWS Audit Manager or AWS Config.
2. Assuming the Well-Architected Tool Automatically Scans Resources: Believing that the Well-Architected Tool automatically runs automated vulnerability scans on EC2 instances or S3 buckets. The Well-Architected Tool is primarily an architectural questionnaire and evaluation framework (though integrations exist to import findings via Trusted Advisor and Control Tower). It does not autonomously execute automated remediation or intrusive network scans.
3. Failing to Execute the HIPAA BAA Before Storing PHI: Assuming that because an AWS service is eligible under HIPAA (e.g., Amazon S3, Amazon RDS), an organization is legally compliant to store PHI immediately. Under the shared responsibility model, an organization must formally review and accept the Business Associate Addendum (BAA) in AWS Artifact prior to processing, storing, or transmitting PHI on AWS.
4. Overlooking Cryptographic Manifest Integrity in Audit Manager: Disregarding the importance of the manifest file when presenting assessment reports to external auditors. The manifest file containing SHA-256 hashes is what legally validates that the automated evidence was collected directly from AWS telemetry without post-capture tampering.