2.3 Classification Models

Quick Answer: Classification models predict discrete categories like spam/not spam or disease/no disease. Binary classification has two outcomes; multi-class has three or more. Evaluation metrics include accuracy, precision, recall, and F1-score. The confusion matrix shows true/false positives and negatives.

What Is Classification?

Classification is a supervised machine learning technique that predicts a discrete category (class) for each input. Unlike regression which outputs a number, classification outputs a label.

How to Identify a Classification Problem

Ask yourself: Is the predicted output a category or label?

• Predicting if an email is spam or not → Classification (two categories)
• Predicting a house price → Regression (continuous number)
• Predicting the species of a flower (setosa, versicolor, virginica) → Classification (three categories)
• Grouping customers by behavior → Clustering (unsupervised)

Binary vs. Multi-Class Classification

Binary Classification

Predicts one of two possible outcomes:

Multi-Class Classification

Predicts one of three or more possible outcomes:

The Confusion Matrix

The confusion matrix is a table that shows how well a classification model performs by comparing predictions to actual values. For binary classification:

• True Positive (TP): Model correctly predicts positive (detected spam that was actually spam)
• True Negative (TN): Model correctly predicts negative (correctly identified a legitimate email)
• False Positive (FP): Model incorrectly predicts positive (flagged a legitimate email as spam) — also called a "Type I error"
• False Negative (FN): Model incorrectly predicts negative (missed actual spam) — also called a "Type II error"

On the Exam: Questions may present a scenario and ask which type of error is more concerning. For medical screening, false negatives (missing a disease) are typically worse than false positives (incorrectly flagging someone for further testing).

Classification Evaluation Metrics

Accuracy

The proportion of all predictions that are correct.

Formula: (TP + TN) / (TP + TN + FP + FN)

Interpretation: "What percentage of all predictions were correct?"

Limitation: Accuracy can be misleading with imbalanced datasets. If 95% of emails are not spam, a model that always predicts "not spam" has 95% accuracy but catches zero spam.

Precision

The proportion of positive predictions that are actually correct.

Formula: TP / (TP + FP)

Interpretation: "Of all the items I flagged as positive, how many actually were positive?"

When precision matters: When false positives are costly. Example: A spam filter with high precision rarely flags legitimate emails as spam.

Recall (Sensitivity)

The proportion of actual positives that the model correctly identifies.

Formula: TP / (TP + FN)

Interpretation: "Of all the actual positive items, how many did I find?"

When recall matters: When false negatives are costly. Example: A medical screening with high recall catches most actual disease cases.

F1-Score

The harmonic mean of precision and recall — a balanced measure.

Interpretation: A single metric that balances precision and recall. Useful when you need both to be high.

Precision vs. Recall Trade-off

There is often a trade-off between precision and recall:

• Increasing precision (fewer false positives) often decreases recall (more false negatives)
• Increasing recall (fewer false negatives) often decreases precision (more false positives)

Example: Medical screening

• High recall priority: You want to catch EVERY case of a disease, even if some healthy people are flagged for follow-up tests (accept more false positives to minimize false negatives)
• High precision priority: You only want to diagnose someone as sick when you are very confident (accept missing some cases to minimize false alarms)

On the Exam: Questions may describe a scenario and ask which metric is most important. Medical screening = recall (catch every case). Spam filter = precision (don't block legitimate emails). When both matter equally = F1-score.

Worked Example: Computing Metrics from a Confusion Matrix

Imagine a fraud detector evaluated on 1,000 transactions produces this confusion matrix:

From these four numbers you can derive every metric:

• Accuracy = (TP + TN) / total = (32 + 940) / 1000 = 97.2% — but this is flattered by the huge legit majority.
• Precision = TP / (TP + FP) = 32 / 52 = 0.62 — when the model cries fraud, it is right 62% of the time.
• Recall = TP / (TP + FN) = 32 / 40 = 0.80 — it catches 80% of actual fraud.
• F1-score balances precision and recall into a single number (here ≈ 0.70).

This illustrates why accuracy alone is misleading on imbalanced data: a useless model that always predicts "legit" would still score 96% accuracy while catching zero fraud (recall 0%).

Choosing the Right Metric for the Business Cost

Multi-Class Evaluation and Probability Outputs

For multi-class problems, the confusion matrix grows to N×N (one row and column per class), and precision/recall are calculated per class then averaged. Classification models also output a probability or confidence score per class — for example "90% cat, 8% dog, 2% bird." The predicted label is simply the highest-probability class, and you can raise the decision threshold to trade recall for precision.

On the Exam: Two reliable patterns. First, imbalanced data + high accuracy is a red flag — the right metric is precision, recall, or F1. Second, when the question describes which mistake is more harmful, map it: harmful false negatives → recall; harmful false positives → precision; both harmful → F1-score. You will never have to do the arithmetic, but understanding what each number means lets you pick the correct option instantly.