Supervised, Unsupervised & Reinforcement Learning
Key Takeaways
- Supervised learning needs labeled examples, and the labels — not the algorithm — are usually the schedule and budget risk; classification predicts a category, regression predicts a number.
- Unsupervised learning has no ground truth, so its output must be validated by a domain SME against a business use rather than scored against a held-out answer key.
- Reinforcement learning learns from a reward signal by trial and error and normally needs a simulation environment; it powers the Goal-Driven Systems and Autonomous Systems patterns.
- Generative AI shifts work from training a model to grounding, prompting and evaluating a pre-trained one; evaluation becomes rubric-based human review on a fixed prompt set, not a single accuracy number.
- The strongest signal for machine learning is a task whose patterns are too complex or too changeable to write as fixed rules; if the logic is known, stable and enumerable, deterministic rules are cheaper, auditable and instantly changeable.
Three Ways a Model Learns
Machine learning is not one thing. Which of the three learning approaches your team proposes decides whether you must buy or create labeled data, how long Phase 3 Data Preparation and Phase 4 Model Development will run, and — the part project managers most often miss — whether the result can be evaluated objectively at all. You do not need to write the code. You do need to know which approach is on the table before you agree to a schedule.
Supervised Learning
The team gives the algorithm examples that already carry the right answer — a label — and the model learns to reproduce it on new cases. It takes two shapes. Classification predicts a category: fraud or not fraud, which of eleven claim types, which diagnosis code. Regression predicts a number: next month's demand, days until this pump fails, expected claim cost.
Supervised learning is the workhorse of enterprise AI, and it is also where projects quietly bleed schedule, because the labels are the deliverable nobody budgeted. Ask three questions during Phase 2 Data Understanding:
- Do labels exist at all? "We have ten years of claims" is not the same as "we have ten years of claims each marked as fraudulent or clean."
- Who will create them, and at what cost? Labeling 20,000 images or 5,000 clinical notes is a staffed workstream with its own schedule, quality process and budget line — not a task.
- Are the labels trustworthy? Two annotators disagreeing on the same record is a measurable problem you can quantify and fix with a coding guide. Worse is a label that encodes past behavior: your "fraud" flag only marks the fraud investigators happened to catch, and a "successful hire" label reproduces whoever the company used to hire.
What you get in return for that cost is objectivity. A supervised model can be scored on data it has never seen, so Phase 5 Model Evaluation has a defensible, numeric answer.
Unsupervised Learning
No labels. The algorithm finds structure on its own. Clustering groups similar records — customer segments, equipment behaving alike, documents about the same topic. Dimensionality reduction compresses many variables into a few that carry most of the signal. It is often the cheapest thing to run because you skip labeling entirely, and it is genuinely useful during Phase 2 for seeing what is actually in the data before you commit to a solution.
The catch is evaluation. With no ground truth there is no held-out answer key, so "is this segmentation good?" is a judgment made by a domain expert, not a score. Plan for that: put a subject matter expert in the loop, schedule review sessions, and write acceptance in business terms before the work begins — for example, "the segments are distinguishable enough that marketing can write a different offer for each." Never let an unsupervised deliverable be signed off on the strength of a colorful chart.
Reinforcement Learning
The system learns by acting, receiving a reward signal, and adjusting — trial and error against an objective rather than imitation of labeled examples. It needs a simulation environment or another safe place to explore, because early attempts are bad by design and you cannot let a learning agent practice on live pricing or live equipment. Reinforcement learning is what typically sits behind the Goal-Driven Systems and Autonomous Systems patterns.
Three implications for you. The simulator is frequently a larger build than the model itself, and it belongs in the estimate. Specifying the reward is a business decision with real consequences, because an agent rewarded only for margin will find margin at the expense of churn. And validation runs in shadow mode — the agent proposes, humans decide — before it is allowed to act on anything.
Generative AI and Large Language Models
Generative AI is a distinct case with different economics, evaluation and risks. Instead of training a model on your labeled data, you usually take a large language model (LLM) somebody else pre-trained and adapt it — through prompting, retrieval from your own approved content, or fine-tuning on a modest set of examples. That inverts the usual project shape: Phase 3 gets lighter on labeling and heavier on curating trustworthy, current, permissioned source content, while Phase 5 Model Evaluation gets considerably harder.
There is no single accuracy number for "write a good claim summary." Evaluation becomes rubric-based and human-judged: a fixed set of representative prompts, defined quality criteria, and trained reviewers — all of which you must schedule and staff. The risk properties differ too. Output can be fluent and wrong. The same prompt can produce different answers on two runs. Sensitive data can leave the organization through a third-party service. And a vendor can change the model version underneath you, shifting behavior with no change on your side. Treat version pinning, a regression prompt set, and a data-handling agreement as Phase 1 scope items.
Choosing Between Approaches
| Approach | What the data must look like | Typical evaluation | Project-management implication |
|---|---|---|---|
| Supervised — classification | Historical records each carrying the correct category | Precision, recall and a confusion matrix on held-out data | Labeling is a staffed workstream; objective sign-off is achievable |
| Supervised — regression | Historical records with the numeric outcome recorded | Average size of the error against actuals on held-out data | Needs long, clean history; plan for relationships that shift |
| Unsupervised | Records with useful attributes; no labels required | Judgment of a domain SME against a stated business use | Cheap to run, expensive to agree on; define acceptance in business terms up front |
| Reinforcement | A defined reward, explicit constraints, and a simulator or safe sandbox | Objective attained under constraints in simulation, then shadow mode | The simulator can exceed the model in cost; reward design is a business decision |
| Generative / LLM | Curated, current, permissioned content to ground answers on | Rubric scoring by trained human reviewers on a fixed prompt set | Staff the evaluation, pin model versions, control what data leaves the organization |
The Trap: Machine Learning Where Rules Would Do
The exam punishes reaching for machine learning by default. Deterministic rules are cheaper to build, instantly changeable, auditable line by line, and they never drift. If the logic is known, stable and enumerable — "flag any invoice over 10,000 dollars without a purchase order," "route Spanish-language calls to the Madrid queue" — write the rule. Adding a model there buys you cost, latency, an explainability problem and a permanent monitoring obligation in exchange for nothing.
The strongest signal for machine learning is the opposite condition: the task depends on patterns in data that are too complex to express as fixed rules, or that keep changing, and enough historical data exists to learn them. Fraud qualifies, because adversaries adapt faster than any rule committee. Forecasting demand across thousands of items qualifies. Reading handwriting qualifies. Approving an expense against a fixed policy does not. When you review a proposed solution in Phase 1, ask the team to state the rule out loud. If they can state it, you do not need a model.
A team proposes a supervised classification model to route support tickets into 40 categories. During Phase 2 Data Understanding you learn that the historical "category" field was filled in inconsistently by agents and that 30% of tickets are tagged "Other." What should the project manager do next?
An operations director asks for "AI" to choose between three shipping carriers. The choice is fully specified in a two-page policy based on weight, destination zone and service level, and the policy is rewritten whenever carrier contracts are renegotiated. What should the project manager recommend?
A team proposes an LLM-based assistant that drafts customer email replies from an internal knowledge base. The executive sponsor asks what accuracy target you will commit to. What is the best response?