ISTQB AI Testing Specialist (CT-AI)

Which ISO standard extends ISO/IEC 25010 with quality characteristics specific to AI-based systems?

• AISO/IEC 25010
• BISO/IEC 25012
• CISO/IEC 25059
• DISO/IEC 27001

Which of the following is NOT one of the AI-specific quality characteristics or sub-characteristics highlighted in ISO/IEC 25059 (as used in ISTQB CT-AI)?

• AFunctional adaptability
• BAI robustness
• CMaintainability
• DIntervenability

What term describes the phenomenon where a capability stops being considered AI once it becomes commonplace?

• AAI winter
• BAI effect
• CAI singularity
• DAI saturation

Which type of machine learning trains a model using labeled input-output pairs?

• AUnsupervised learning
• BSupervised learning
• CReinforcement learning
• DSelf-supervised learning

Which ML task assigns inputs to one of a fixed set of discrete categories?

• ARegression
• BClustering
• CClassification
• DDimensionality reduction

In a typical ML workflow, what is the purpose of a validation set (separate from train and test sets)?

• ATo produce the final published accuracy figure
• BTo tune hyperparameters and select the best model without leaking the test set
• CTo train the model
• DTo measure data drift in production

What is k-fold cross-validation primarily used for?

• AReducing training time
• BProducing a more reliable performance estimate from limited data
• CIncreasing the size of the training set
• DEliminating class imbalance

What is data leakage in a machine learning context?

• ASensitive data being exposed to attackers
• BInformation from outside the training set inadvertently influencing the model
• CLoss of training data due to disk failure
• DMemory leaks during model training

Which sampling technique preserves the class distribution between train and test sets?

• ARandom sampling
• BStratified sampling
• CConvenience sampling
• DBootstrap sampling

What is the typical first phase of the ML workflow?

• AHyperparameter tuning
• BData acquisition
• CModel deployment
• DDrift monitoring

In a confusion matrix for a binary classifier, what does a 'False Positive' represent?

• AThe model correctly predicted the positive class
• BThe model predicted positive but the true class was negative
• CThe model predicted negative but the true class was positive
• DThe model correctly predicted the negative class

Which formula correctly defines precision?

• ATP / (TP + FN)
• BTP / (TP + FP)
• C(TP + TN) / (TP + TN + FP + FN)
• D2 * (Precision * Recall) / (Precision + Recall)

Which formula correctly defines recall (sensitivity)?

• ATP / (TP + FN)
• BTP / (TP + FP)
• CTN / (TN + FP)
• DFP / (FP + TN)

A medical screening test for a serious cancer should be tuned to favor which metric?

• APrecision
• BRecall
• CAccuracy
• DSpecificity

What does the F1 score represent?

• AThe arithmetic mean of precision and recall
• BThe harmonic mean of precision and recall
• CThe geometric mean of precision and recall
• DPrecision divided by recall

For a heavily imbalanced dataset (99% negative class), which metric is most misleading?

• AAccuracy
• BPrecision
• CRecall
• DF1 score

What does ROC-AUC measure?

• AThe area under the precision-recall curve
• BThe area under the receiver operating characteristic curve, indicating ability to rank po…
• CThe accuracy at a fixed decision threshold
• DThe mean squared error of probability outputs

Which regression metric squares the errors, giving more weight to large mistakes?

• AMAE
• BMSE
• CR²
• DMAPE

What does an R² value of 0.0 indicate?

• AThe model is perfectly accurate
• BThe model performs no better than predicting the mean of the target
• CThe model has zero variance
• DAll errors are zero

What is metamorphic testing especially useful for in AI systems?

• AValidating against a known oracle for every input
• BTesting when an exact ground-truth oracle is unavailable
• CReducing training time
• DGenerating new training data

A classifier predicts 'cat' for an image. After adding imperceptible noise (FGSM perturbation), it predicts 'dog'. What is this an example of?

• AA metamorphic relation
• BAn adversarial example
• CConcept drift
• DClass imbalance

Which of the following is a common deployment strategy where a new model serves a small percentage of live traffic before full rollout?

• AShadow deployment
• BA/B test
• CCanary deployment
• DBlue-green deployment

A model's input feature distribution gradually shifts in production but the relationship between inputs and outputs is unchanged. This is best described as:

• AConcept drift
• BData drift
• CAdversarial attack
• DLabel leakage

Which statistic is commonly used to detect distributional shift between training and production data?

• APopulation Stability Index (PSI)
• BMean absolute error
• CR-squared
• DF1 score

Which fairness metric requires equal true-positive rates across protected groups?

• ADemographic parity
• BEqual opportunity
• CEqualized odds
• DCalibration

Which explainability technique attributes a prediction to its input features using cooperative game theory?

• ALIME
• BSHAP
• CPartial dependence plot
• DPermutation importance

A test pyramid for ML systems typically includes which three layers?

• AUnit, integration, end-to-end
• BData tests, model tests, infrastructure tests
• CManual, automated, exploratory
• DBlack-box, white-box, gray-box

What is the primary purpose of MLOps continuous training (CT)?

• ATo run the same training code repeatedly
• BTo re-train models automatically when data or performance triggers fire
• CTo verify training infrastructure availability
• DTo compress models for edge deployment

Which tool is commonly used to track ML experiments, parameters, metrics, and model artifacts?

• AWireshark
• BMLflow
• CJMeter
• DSelenium

Which type of bias arises when training data is collected disproportionately from one subgroup?

• ASampling bias
• BConfirmation bias
• CAutomation bias
• DAlgorithmic bias

In neural network testing, what does 'neuron coverage' measure?

• AThe percentage of test cases that pass
• BThe fraction of neurons that are activated by the test set above a threshold
• CThe number of neurons in the network
• DThe precision of the network on the test set

A classifier is trained on perfectly balanced 50/50 data but deployed where the positive class is 5%. What problem is most likely?

• AConcept drift
• BClass prior shift causing miscalibrated probabilities
• CMemory exhaustion
• DLoss of training data

When red teaming a generative AI / LLM-based system, which risk is most important to probe for?

• AThat responses will always be identical across runs
• BHarmful, biased, or policy-violating outputs that normal functional tests may miss
• CThat the model will use less GPU memory
• DThat unit tests of preprocessing code will fail

A model performs well on training data but poorly on the test set. What is this called?

• AUnderfitting
• BOverfitting
• CData drift
• DBias amplification

Which statement best describes 'transparency' as an AI quality characteristic?

• AThe model runs in a public cloud
• BStakeholders can understand how decisions are made
• CAll training data is open source
• DThe model is fully deterministic

Which is an example of 'functional adaptability' as defined in ISO/IEC 25059?

• AA model that requires manual data labeling for every prediction
• BA deployed recommender that updates its predictions as user behavior changes after release
• CA model that always asks for confirmation before any output
• DA spreadsheet using a fixed regression formula