9.5 Compliance Artifact, Audit Manager, Macie, and Policy Evidence

Evidence, not assumptions

Compliance conversations often begin with a simple question: can we prove that this AI workload follows policy? A confident verbal answer is not enough. A team needs evidence about AWS service assurances, account configuration, data classification, access control, monitoring, review cadence, and remediation. AI raises the stakes because prompts, retrieved documents, outputs, logs, and model actions can all become part of the compliance story.

AWS Artifact is a portal for AWS compliance reports and certain agreements. It helps organizations review AWS third-party audit reports and service assurance documents when assessing AWS as a cloud provider. A practitioner should understand that Artifact evidence is about AWS controls and agreements. It does not prove that the customer's own AI application is configured correctly, uses approved data, or reviews risky outputs.

AWS Audit Manager helps automate evidence collection and organize it against frameworks. It can collect evidence from AWS sources and help teams manage assessment workflows. This is useful when an organization needs to show that controls are being monitored over time. Audit Manager does not replace control ownership. If a policy says model invocation logs must be reviewed weekly, the team still needs a real process and accountable owner.

Amazon Macie helps discover sensitive data in Amazon S3 using machine learning and pattern matching. For AI workloads, Macie can be useful when S3 buckets contain source documents, training datasets, exported logs, transcripts, or generated outputs. Findings can identify personal data or sensitive patterns that should be remediated, restricted, encrypted, excluded from a knowledge base, or reviewed before use.

Policy evidence should be mapped to the AI lifecycle. Before launch, the team should document the use case, approved data sources, model selection rationale, access decisions, guardrails, retention rules, human review requirements, and cost limits. During operation, the team should collect logs, findings, configuration records, review notes, incident tickets, and change approvals. At retirement, the team should document data deletion, access removal, and model or knowledge base decommissioning.

Data classification is a major compliance dependency. A team cannot govern what it has not classified. If a knowledge base indexes S3 documents, the organization should know whether those documents contain public, internal, confidential, regulated, or customer data. Macie can help discover sensitive data in S3, but business owners must decide what findings mean for policy. A bucket with sensitive data might require stricter access, exclusion from AI retrieval, redaction, or a different workflow.

Compliance also includes evidence of review. For a low-risk internal drafting assistant, review may mean periodic access review, cost review, and prompt-quality checks. For a regulated customer workflow, review may include documented approvals, human review of outputs, incident response testing, retention review, and audit trails for model changes. The key is matching evidence depth to business risk.

Audit readiness checklist:

• Identify the compliance framework, internal policy, or customer requirement being supported.
• Separate AWS provider evidence from customer workload evidence.
• Use AWS Artifact for AWS reports and agreements where appropriate.
• Use Audit Manager when ongoing evidence collection and assessment workflow are needed.
• Use Macie to discover sensitive data in S3 sources, logs, exports, and document stores.
• Use CloudTrail, CloudWatch, and Config to support activity, operations, and configuration evidence.
• Assign owners for findings, exceptions, remediation deadlines, and review cadence.
• Keep evidence linked to the AI use case, not only to generic account controls.

Scenario: a healthcare-adjacent company wants to summarize support tickets with AI. Artifact may support vendor risk review of AWS. Macie can scan S3 buckets that contain ticket exports for sensitive information. Audit Manager can help collect evidence for selected controls. The company still needs its own policy about which tickets may be processed, who can view outputs, and whether human review is required.

Scenario: a bank asks whether an AI assistant has compliance evidence. The best answer is not just a screenshot of the assistant. Evidence should include approved use case documentation, IAM permissions, data classification, logs, guardrail settings, source approval, retention settings, and review records. If the assistant can perform actions, evidence should include tool permissions, confirmation rules, and action logs.

Other AWS security services can contribute context. Amazon Inspector can help identify software vulnerabilities in supported workloads that host AI application components. Trusted Advisor can highlight certain account-level best practice checks. These services are useful signals, but they do not replace AI-specific governance around prompts, data, outputs, and model access.

For AIF-C01, remember the boundary. You are not expected to implement a compliance framework. You should recognize which AWS service helps provide provider reports, which helps organize evidence, which helps find sensitive S3 data, and why evidence must be tied back to customer-owned policies and workload controls.