8.3 A.7 Data for AI Systems
Key Takeaways
- Annex A.7 governs data used to develop, evaluate, and enhance AI systems—acquisition, quality, provenance, and preparation under defined responsibilities.
- Data acquisition controls address lawful/authorized collection, licensing, consent where relevant, and fitness of sources—not only “we scraped a lot.”
- Data quality for AI is multi-dimensional (accuracy, completeness, representativeness, timeliness, consistency) and must be measured for the intended task.
- Provenance and lineage enable auditors to answer where data came from, how it was transformed, and which model versions consumed which snapshots.
- Strong audit evidence includes datasheets, lineage graphs, quality metrics, access controls, and preparation pipelines that are versioned and reviewable.
8.3 A.7 Data for AI Systems
Auditor focus: Models inherit the virtues and vices of their data. Annex A.7 (Data for AI systems) asks whether the organization governs training, validation, test, and enhancement data with the same seriousness as production code—quality, origin, preparation, and rights.
Control objective and typical A.7 map
| Control (common map) | Intent | Primary auditor questions |
|---|---|---|
| A.7.2 Data for AI systems | Overall data management for AI development/enhancement | Are policies, roles, and inventories defined for AI data assets? |
| A.7.3 Data acquisition | Obtain data appropriately | Legal basis, licenses, contracts, collection methods, suitability? |
| A.7.4 Data quality | Fit for purpose | Defined dimensions, metrics, thresholds, remediation? |
| A.7.5 Data provenance | Origin and lineage | Can the chain of custody and transformations be reconstructed? |
| A.7.6 Data preparation | Clean, label, transform under control | Versioned pipelines, labeling QA, documented transforms? |
A.7 interfaces with A.6 (especially development and V&V data), A.5 (impact—who is affected by data gaps), A.4 (data/compute resources), privacy/security (often ISMS), and A.10 (third-party data and foundation-model training corpora claims).
Data for development and enhancement
AI systems consume data at multiple points:
- Training / fine-tuning data that shapes parameters.
- Validation / test data for V&V (should be controlled against leakage).
- Operational / feedback data used for monitoring and periodic enhancement.
- Synthetic or augmented data, which still needs provenance and quality rules.
Inventory expectation: Data assets linked to AI systems (which model versions use which datasets/snapshots). Without inventory, sampling collapses and shadow datasets proliferate.
Enhancement risk: Continuous learning or frequent retrain can silently change behavior. Auditors check whether new data batches pass the same acquisition, quality, and preparation controls before they affect production (gate back to A.6 deploy).
Data acquisition (A.7.3)
Acquisition controls address how data enters the AI estate:
| Source type | Evidence to request |
|---|---|
| Internal operational systems | Data sharing agreements, purpose limitation, minimization |
| Customer / user data | Privacy notices, consent or other lawful basis, retention |
| Licensed third-party datasets | License terms, permitted use (train vs. redistribute), restrictions |
| Public web / scrape | Legal review, robots/ToS analysis, sensitive content filters |
| Human subjects / special categories | Ethics approval, heightened safeguards |
| Vendor / partner feeds | Contractual quality and rights clauses; security of transfer |
Red flags: “Public data is free for any ML use” without review; training on customer content despite contractual prohibition; no record of who approved a new source; personal data in prompts logged without governance.
Auditor stance: You are not the general counsel, but you test whether a process exists, whether required reviews occurred for sampled datasets, and whether exclusions in the SoA are honest if acquisition control is claimed N/A.
Data quality for AI systems (A.7.4)
Quality is task-relative. A dataset can be complete for billing yet unsuitable for fair credit decisioning.
| Dimension | Example AI concern |
|---|---|
| Accuracy | Wrong labels; sensor error |
| Completeness | Missing fields correlated with subgroups |
| Consistency | Conflicting schemas across sources |
| Timeliness | Stale labels after concept change |
| Representativeness | Under-coverage of populations in production |
| Uniqueness / duplication | Leakage; inflated metrics |
| Integrity | Unauthorized alteration of labels |
Evidence: defined quality criteria per dataset class; measurement results (before train and periodically); defect handling; exclusion of bad batches; link between quality failures and model risk acceptance.
Trap: Single overall accuracy of a model is not a data quality program. Ask for dataset-level metrics and labeling QA (e.g., inter-annotator agreement where labels are human).
Data provenance (A.7.5)
Provenance answers: Where did this come from, what happened to it, and which models used it?
Strong lineage package:
- Source system / supplier and acquisition date.
- Raw storage location and access controls.
- Transformation pipeline version and parameters.
- Labeling vendor/version and QA outcomes.
- Snapshot or feature-store version IDs consumed by training run X.
- Link from training run X → model version Y → production deploy Z.
| Weak provenance | Strong provenance |
|---|---|
| “From the data lake” | Named pipelines, job IDs, dataset semantic versions |
| Labels with no author | Annotator guidelines + sample audit |
| Manual CSV email | Controlled registry with checksums |
| Unknown scrape date | Documented crawl window and filters |
Without provenance, organizations cannot investigate poisoning, bias sources, or regulatory data-subject issues—and cannot support reproducible V&V.
Data preparation (A.7.6)
Preparation includes cleaning, normalization, feature engineering, tokenization, anonymization/pseudonymization, balancing, and labeling. Controls should ensure:
- Documented recipes (not only tribal notebook knowledge).
- Version control of preparation code.
- Separation of train/validation/test splits with leakage prevention.
- Labeling guidelines, training for annotators, quality sampling.
- Privacy-preserving transforms applied before broad ML access where required.
- Review when preparation changes (can alter model behavior as much as architecture).
Sampling: Pick one labeled dataset. Review guidelines, sample labeled items vs. guidelines, QA metrics, and whether disputed labels have an escalation path. Pick one feature pipeline and confirm the production features match training assumptions (train–serve skew is both A.7 and A.6.2.6 operation and monitoring).
Auditor evidence catalog
| Evidence type | What you learn |
|---|---|
| Datasheets for datasets / data cards | Intended use, composition, collection, limitations |
| Lineage / feature store records | Provenance and reuse |
| Quality dashboards & reports | Whether A.7.4 is operational |
| Licenses, DPAs, consent records | Acquisition legitimacy |
| Access control & DLP logs | Who can alter training data |
| Preparation pipeline repos | Reproducibility of transforms |
| Data incident records | Poisoning, leakage, mislabel events → Clause 10 |
Interview targets
- Data owner / steward for AI datasets.
- ML engineer who last retrained.
- Privacy/legal reviewer for a new source.
- Labeling lead or vendor manager.
Scenario: Generative system fine-tuned on mixed internal documents. No license review for embedded third-party content; no datasheet; quality defined only as “enough tokens.” Findings likely under A.7.2–A.7.5, with possible A.10 and privacy ISMS linkage. Severity rises if outputs can reproduce confidential or personal data.
Exam tip: Do not confuse A.7 (data for AI) with A.6 life cycle or with A.8 information for interested parties. Data quality failures that cause unfair outcomes also connect to section 8.4 audit techniques—but the normative home for data controls is A.7.
Which control group is primarily responsible for governing acquisition, quality, provenance, and preparation of data used to train and enhance AI systems?
An organization cannot identify which raw sources and transformation jobs produced the snapshot used to train the current production model. Which A.7 theme is most directly deficient?
When sampling data acquisition for a licensed third-party training corpus, which evidence set is most relevant?
Why is inter-annotator agreement (or equivalent labeling QA) useful audit evidence under A.7 data preparation/quality?