11.2 ML Model Documentation and Review

Why documentation is the main instrument, not a courtesy

In conventional software you can often open a repository, read a function, and argue about what it does. Machine learning systems (MLS) do not give testers that luxury. The artifact that makes predictions is machine-generated. Weights and graphs are not a readable specification. Many models are black-box in practice: even when you know the architecture, you cannot narrate why this example scored 0.81. Behavior is data-dependent, so two teams that type similar training code can ship different systems if their corpora differ. Models are frequently updated, which means last quarter's tribal knowledge is already stale.

For those reasons, documentation is the main tool developers, testers, and regulators have for understanding, evaluating, and trusting an AI-based system. Learning objective AI-6.1.2 (K2) asks you to explain the purpose and focus of reviewing that documentation. The purpose is not to produce a pretty PDF. The purpose is to make the system transparent enough that a skilled outsider can trace behavior, decision logic, and data lineage through the lifecycle: where the data came from, how it was labeled, which algorithm was chosen, how the model was evaluated, where it is allowed to be used, and how it will be watched after it ships.

If that trail is missing, testers cannot design bias tests, drift monitors, or rollback drills. Regulators cannot tell whether a high-risk system was even described honestly. Operators cannot know which threshold was supposed to trigger human review. Documentation review is therefore a core test activity, not an editorial afterthought.

Transparency, and why the EU AI Act cares

Transparency here is a quality idea: appropriate information about the AI-based system is communicated to the people who need it. It is not a claim that the neural network has become magically interpretable. It is a claim that stakeholders can follow the story of the system—inputs, training data, known limits, intended users, and how outputs should be read.

That story is increasingly a legal obligation. The EU AI Act emphasizes transparency obligations: clear documentation of model decisions, limitations, and interpretability measures. For high-risk systems, passing a documentation audit is often a prerequisite for deployment. For every system, a thorough review still helps verify quality. You do not need to recite article numbers on this exam. You do need the tester implication: if the file cannot support an audit, the model is not ready, even if a notebook metric looks flattering.

Standardized documentation also improves communication inside the building. A data scientist, a test lead, and a risk officer can disagree productively only if they are looking at the same description of intended use, evaluation, and limits. Ad hoc wiki pages that mention the model without a version hash do not do that job.

Two frameworks you must be able to contrast

You are not required to invent a documentation standard from scratch. Two widely cited frameworks show up as named examples.

Model Cards give a concise overview of a trained model. Typical contents include intended uses, evaluation results, and ethical considerations. A card is the model's identity sheet: who it is for, what it was measured on, where it should not be used, and which fairness or harm issues the authors already know about. If a vendor hands you weights and a one-line README, you do not have a model card.

Datasheets for Datasets describe the data rather than the fitted model. They offer a standardized way to record motivation (why the set was created), composition (what is in it), collection (how it was gathered), and uses (what it is and is not for). Because MLS are data-dependent, a beautiful model card sitting on a mystery corpus is still an incomplete story. Testers who cannot tell how labels were produced cannot judge whether an evaluation set is independent or contaminated.

Use the two artifacts together. The datasheet explains the fuel. The model card explains the engine that was built from that fuel, how it was scored, and which ethical limits apply. Neither artifact replaces the fuller checklist below; they are starting patterns that keep teams from improvising a different outline every sprint.

A practical checklist, section by section

Comprehensive ML model documentation is structured so developers and testers can walk it as a completeness, clarity, and testability checklist. Teach yourself the buckets, then the questions each bucket must answer.

General. Identifiers, a human-readable description, developer or owner, version, date, contact, license, and hardware needs. Without identifiers and version you cannot say which artifact you tested. Without license and hardware notes you cannot tell whether a lab GPU run is even a legal or feasible replica of production.

Design. Assumptions and technical decisions, including the ML algorithm. If the team assumed labels are noise-free, or that class priors will stay stable, those assumptions belong here. Testers later turn them into risks: noisy labels, prior shift, an algorithm that cannot represent the required function.

Usage. Intended purpose, primary and secondary uses, users, any self-learning approach, bias, ethics, safety, transparency, thresholds, platform, data drift, and concept drift. This is the section that tells you who may rely on the score and what happens when the world moves. Self-learning or continually trained systems need extra operational detail; a locked model still needs an explicit statement that it will not update itself in production.

Datasets. Features, source, collection, availability, preprocessing, use, content, labels, size, privacy, security, bias and fairness, and restrictions. Testers look for whether personal data was stripped or justified, whether minority groups are described, and whether the evaluation set is actually a sibling of the training set rather than a reshuffle of it.

Testing. Test dataset details, independence of testing, test results, and test activities (functional, adversarial, and others actually performed). Independence is not a slogan. If the test set leaked into training, the reported numbers are not evidence.

Functional. Measures, validation dataset, thresholds, and actual performance. A recall target without a threshold and without the set it was measured on is not an acceptance criterion.

Non-functional. Scalability, reliability, availability, performance efficiency (latency, resource utilization), maintainability, and AI robustness. MLS fail in production from timeouts and memory as often as they fail from a wrong class. If the document is silent on latency, performance testing has no agreed bar.

Operational. Deployment plan, deployment environment, computational resources, monitoring metrics and alerts, retraining strategy, model update and rollback plan, deprecation plan, security including adversarial risks, and explainability methods. This is the section that turns a trained file into a living service. Rollback and deprecation are easy to skip and expensive to invent during an incident.

What a documentation review is trying to find

Walking the checklist is not a formatting exercise. Reviewers use it to pursue five goals.

First, identify missing information, inaccuracies, and inconsistencies. Missing: no hardware note, no license, no statement of whether the model is self-learning. Inaccurate: a claimed test-set size that does not match the attached file. Inconsistent: the usage section forbids medical advice while the evaluation section brags about a diagnosis F1-score. Any of those defects is a test finding.

Second, improve clarity and readability. A regulator or a night-shift operator must be able to find the threshold without decoding internal slang. If two names are used for the same version, clarity has already failed.

Third, enhance maintainability by showing where the documentation itself needs improvement. Models change. If the only copy of the preprocessing steps lives in a departed contractor's notebook, the next retrain will be a guess. Review comments that say this section cannot be updated without archaeology are maintainability findings.

Fourth, provide sufficient information for test and deployment activities. Could you design an independent test set from what is written? Could you stand up the production environment? Could you roll back? If the answer is no, the document is not test-complete, even if it is long.

Fifth, verify that relevant regulatory requirements have been met. For a high-risk system under the EU AI Act transparency story, that often means the documentation can survive an audit before deploy. For other regimes the details differ; the tester habit does not: match the claimed obligations against what the file actually contains.

A worked review on a vendor model

Suppose a vendor delivers an image model for warehouse defect detection. The packet includes a short Model Card that lists intended use (flag damaged cartons on a belt), an overall accuracy number, and a sentence about fairness. There is no Datasheet. Hardware is described as a modern GPU. The usage section is silent on self-learning. Testing mentions a test set but not whether it is independent of training. Operational text says monitor the model with no metric, no alert, and no rollback.

A competent review does not start by arguing about the accuracy number. It starts by listing holes. Without a datasheet you cannot trace data lineage or label quality. Without versioned identifiers you cannot be sure which weights you will deploy. Without independence language you cannot trust the functional measures. Without rollback and monitoring you cannot operate the system. Those are documentation defects, and they block a high-risk-style audit even if the demo video looks smooth.

The same review can also catch inconsistencies: a card that says not for safety-critical decisions while the deployment plan wires the model's output to an automatic stop of the belt. That contradiction is more important than a missing comma in the license field.

On the exam, keep the logic short. MLS are hard to inspect as source, so documentation carries the understanding. Transparency is the ability to trace behavior, logic, and data lineage. Named frameworks help, but a review still walks a full checklist and then asks whether the file is complete, consistent, clear, maintainable, testable, deployable, and fit for the regulatory bar that applies.