4.3 Acceptance Criteria for AI-Based Systems

Why AI acceptance is rarely a single tick

When you evaluate an AI-based system you still care about functional and non-functional quality. ISO/IEC 25010 and ISO/IEC 25059 give the vocabulary. For the characteristics that ISO/IEC 25059 adds or adjusts—and for safety—acceptance often has to be statistical, probabilistic, or threshold-based rather than a single yes-or-no outcome on one input.

That shift is the extra testing challenge in CT-AI v2.0 learning objective AI-2.2.1 (K2). You must give examples of acceptance criteria that a tester could actually measure. Vague wishes such as the model should be fair and robust are not acceptance criteria. A usable criterion names the characteristic, a metric, a numeric threshold, a time bound or evaluation window, and the evidence (dataset, replay, drill, red team) that will be judged.

The stories below teach the same ideas as the syllabus examples, rewritten as testable scenarios so you can invent new ones on the exam instead of reciting a grid. Numbers in this section are teaching fixtures from that example set, not field measurements of a live product and not a claim that OpenExamPrep assessed any device.

The five-part pattern testers should write

Use this pattern every time you turn a quality characteristic into an acceptance line:

1. Name the ISO/IEC 25059 characteristic (or safety).
2. Name the metric (accuracy, recall, seconds, percent of recommendations, fairness statistic, red-team pass rate).
3. Set a threshold (95%, 0.5 s, 40%, 99.9%).
4. Bound time or conditions (after three documentaries in one session; while the vulnerability database is down for 30 seconds; within 0.15 seconds of detection).
5. State what happens on fail (block release, degrade mode, human veto).

If any of those five are missing, the criterion will collapse into debate during acceptance testing. Two testers should be able to run the same evidence pack and reach the same pass or fail.

AI functional correctness

Correctness criteria accept that some errors will occur and then cap them. You consider correct outputs and incorrect outputs against a threshold.

Teachable examples in this style:

• An image-recognition system used in a defined operational domain shall achieve 95% accuracy on the locked evaluation set that represents that domain.
• A manufacturing defect-prediction system shall achieve 90% recall on the agreed defect-holdout, because missed defects are costlier than extra inspections.

Note what is not on this list. A criterion that gives an engine-management system a maximum of 20 seconds to adapt after it crosses an altitude threshold is a functional adaptability criterion, not a correctness one, and it is covered in the next subsection. Correctness criteria cap wrong answers; adaptability criteria bound autonomous adjustment to a changed environment.

Each line is fail-able. Pretty accurate is not. If the image model is 94.7% on the locked set, correctness failed even if a demo looked impressive. If recall is 90% but the holdout was swapped for an easier slice, you did not measure the criterion you wrote.

Functional adaptability

Adaptability criteria describe autonomous post-deployment change in a measurable way.

• An engine-management system shall complete its adaptation within a maximum of 20 seconds when it crosses a specified altitude threshold. Testers replay the altitude crossing and time the adaptation; the clock bounds the autonomous adjustment, it does not measure how often the function is right.
• After a subscriber watches three full-length documentaries in a single session, the streaming homepage shall recommend at least 40% documentaries, without a human shipping a new model between the third title and the next homepage render.

Testers can script the session, capture the homepage composition, and compute the percentage. If the mix only changes after a weekly batch job that an engineer launches, the criterion was not met as functional adaptability. It was met, at best, as a scheduled content update.

User controllability

Controllability criteria put a human or external agent in the loop in time.

• A supervisor shall be able to take control of an autonomous drone within 0.5 seconds of a distress signal caused by loss of GPS.
• A farm control system shall notify the grower when visual performance degrades by more than 30%, enabling immediate manual override, and shall fully deactivate if degradation exceeds 50% with no user response.

Testers measure takeover latency on a GPS-loss drill and confirm the 30% / 50% notify-and-deactivate path on a degraded-sensor bench. Drop the time bound or the degradation percentages and the criterion becomes untestable. Do not file the drone takeover under intervenability unless the stem is about preventing harm as a security property; the syllabus-style drone story is timely human control of the vehicle.

Transparency

Transparency criteria require specific information to reach stakeholders through named channels.

• The operational dashboard and API shall expose an endpoint that returns the unique version identifier of the currently deployed prediction model and a link to the corresponding documentation.
• Information about a third-party ML model and the provenance of its training data shall meet the relevant company standard. The standard is the oracle; testers check the artifact pack against it.

If the only transparency evidence is a slide in a sales folder, the criterion failed. If the API returns a version string but no documentation link, the dashboard-and-API criterion failed even though some information exists.

AI robustness

Robustness criteria keep AI functional correctness inside bounds when the world is degraded.

• Response time for an AI-based security penetration-alert system shall remain below 1 second when access to the central vulnerabilities database is disrupted for 30 seconds.
• An edge AI device shall transition to a lower-fidelity, reduced-power inference mode instead of crashing when internal temperature exceeds 85°C for 10 continuous seconds.

Testers inject the outage and the thermal condition. Passing on a cool bench with a healthy database does not satisfy these criteria. A 95% clean-set accuracy number, standing alone, is a correctness criterion, not a robustness criterion.

Intervenability

Intervenability criteria give an operator a timed path to prevent harm.

• If a robot breaches its safety zone, the production line shall be capable of shutdown within 0.5 seconds after shutdown is initiated.
• To reduce blackout risk, a power-grid management system shall provide a 30-second confirmation window in which an engineer can veto any AI-proposed action classified as critical before automatic execution.

Time the emergency-stop chain. Prove the veto window exists for critical actions and that expiry behavior is defined. A log message with no actuator is not intervenability. A 30-second window that applies only to low-priority suggestions also fails the critical-action criterion.

Societal and ethical risk mitigation

These quality-in-use criteria use named fairness or harm metrics, not slogans.

• An automated prison-sentencing aid shall not discriminate among racial groups on the specified fairness metric. The project must name the metric and the measurement set.
• A chatbot shall pass an internal red-teaming assessment with a score of 95% or higher, demonstrating refusal to generate content that promotes violence, self-harm, or hate speech.

Testers need the metric definition, the slice of cases, and the scoring rubric. We care about ethics cannot be accepted. A chatbot that is 99% next-token accurate and 80% on the red-team refusal score fails the societal-and-ethical criterion even if correctness looks strong.

Safety

Safety criteria sit beside the 25059 characteristics because harm potential is its own acceptance problem.

• Non-AI components of an AI-based steering-control system shall satisfy ISO 26262-6 at ASIL C.
• For an ML model in a nuclear power plant control context, an explainability tool shall map 100% of the relationships between inputs and outputs with average accuracy no lower than 99.9%.
• Control signals that exceed specified safety limits by more than 10% shall be analyzed and regulated within 0.15 seconds after the safety-monitoring subsystem detects them.

These numbers exist so independent testers can pass or fail a build. They are teaching examples of measurable, time-bounded, metric-named criteria. They are not a claim that OpenExamPrep or any training course stands in for a notified-body assessment.

Notice how safety criteria still follow the five-part pattern. ISO 26262-6 at ASIL C names a standard and a level. One hundred percent mapped relationships at 99.9% average accuracy names coverage and a threshold. The 0.15-second regulation window names a clock. A stem that only says the nuclear model must be explainable is not yet an acceptance criterion.

Tester checklist before you sign the report

• Is the criterion statistical, probabilistic, or threshold-based when the AI behavior is not binary?
• Did you name the metric instead of saying good enough?
• Is there a time bound or a clearly defined evaluation window?
• Could two testers disagree about pass or fail? If yes, tighten the wording.
• Did you attach the criterion to the correct characteristic? Correctness versus robustness, controllability versus intervenability, and transparency versus an unvalidated explanation heatmap are the usual mix-ups.

Rewrite a bad line as practice. The robot should be safe becomes: if the robot breaches its safety zone, the line shuts down within 0.5 seconds after shutdown is initiated, measured on the production e-stop chain during a monthly drill. The first sentence cannot be tested. The second can. That conversion—from wish to stopwatch—is the skill this learning objective samples.