Determine Return on Investment
Key Takeaways
- Model AI benefit from the decision the model improves: decisions per year multiplied by improvement rate multiplied by value per decision, then reduced by a realization factor agreed with finance.
- Total cost of ownership must include data acquisition and labeling, inference compute at production volume, MLOps tooling, monitoring, periodic retraining, human review capacity, and model risk validation — not just the build.
- A model that improves accuracy without changing a decision or an action returns zero, no matter how much the technical metric improves.
- Present AI return on investment as a range with sensitivity on the accuracy-lift assumption; in the illustrative case below, payback moves from roughly 13 months to roughly 27 months on a swing of plus or minus 2 percentage points in one assumption.
- Benefits must be expressed in a metric the business already measures, baselined before deployment, and agreed in writing with finance and a named business benefit owner.
Return on Investment Starts at the Decision
Domain II Task 5 of the Examination Content Outline asks you to determine return on investment (ROI), and the governing discipline is the one that governs scope: money comes from a decision that changes, never from a metric that improves. Model expected benefit as decisions per year x improvement rate x value per decision, then reduce it by a realization factor agreed with finance. If you cannot fill in all three terms, you do not have a business case — you have a research proposal.
A Worked Benefit Model
Every figure below is an illustrative example constructed to demonstrate the arithmetic. None of it is published industry data.
An insurer proposes an AI fraud-referral model on personal auto claims:
- Decision volume: 400,000 claims a year reach the "refer to the Special Investigations Unit (SIU) or pay" decision.
- The binding constraint: the SIU can investigate roughly 16,000 claims a year. That capacity is fixed, so the model's job is to rank better inside a fixed review budget, not to refer more claims.
- Baseline: of today's 16,000 referrals, 25% (4,000) are confirmed fraudulent, at an average avoided payout of USD 6,000.
- Projected lift: the model raises the confirmed-fraud hit rate from 25% to 29% across the same 16,000 investigations — 4 percentage points, or 640 additional confirmed cases.
- Gross annual benefit: 640 x USD 6,000 = USD 3.84 million.
- Realization factor: finance accepts that only about 60% of identified fraud is genuinely avoided or recovered, giving a net annual benefit of USD 2.30 million.
Notice what did the work. Decision volume and the capacity constraint produced the number; an accuracy figure quoted on its own would have produced nothing.
Total Cost of Ownership
Total cost of ownership (TCO) is where AI business cases fail review, because the omitted lines are the recurring ones — infrastructure and ongoing maintenance. Build cost is the smaller half of an AI system's life. The table below continues the same illustrative example.
| Cost line | Year 1 (illustrative USD) | Annual run-rate (illustrative USD) | Commonly omitted |
|---|---|---|---|
| Data acquisition and third-party data licenses | 120,000 | 90,000 | Yes |
| Labeling and subject-matter-expert review time | 85,000 | 25,000 | Yes |
| Data engineering and pipeline build | 260,000 | 60,000 | No |
| Model development effort | 220,000 | 40,000 | No |
| Training compute | 35,000 | 15,000 | No |
| Inference compute at production volume | 18,000 | 22,000 | Yes |
| MLOps (machine learning operations) platform and tooling | 60,000 | 60,000 | Yes |
| Monitoring, drift detection, and alerting | 45,000 | 35,000 | Yes |
| Periodic retraining (twice a year) | 30,000 | 60,000 | Yes |
| Human review and adjudication capacity | 0 | 75,000 | Yes |
| Model risk validation and compliance overhead | 70,000 | 40,000 | Yes |
| Change management, training, documentation | 55,000 | 20,000 | Yes |
| Total | 998,000 | 542,000 |
MLOps tooling, monitoring, retraining, and human review together account for USD 230,000 of the annual run-rate — more than the model development line. A business case built on build cost alone overstates the return by roughly that margin.
Payback, Modeled as a Range
Benefits ramp; costs do not wait. If the solution goes live in month 8 and delivers half its steady-state rate while the workflow stabilizes, Year 1 realizes about USD 479,000 against USD 998,000 of cost — a Year-1 net of roughly negative USD 519,000. From Year 2 the solution runs at USD 2.30 million of benefit against USD 542,000 of run cost, about USD 1.76 million a year or USD 147,000 a month. The Year-1 deficit clears about 3.5 months into Year 2, giving payback at roughly month 15 from project start.
Now move the single assumption nobody can verify before CPMAI Phase 5 (Model Evaluation):
| Assumed lift in SIU hit rate | Net annual benefit (illustrative USD) | Payback from project start |
|---|---|---|
| +2 points (pessimistic) | 1.15 million | about 27 months |
| +4 points (planning case) | 2.30 million | about 15 months |
| +6 points (optimistic) | 3.46 million | about 13 months |
A two-point swing in one unobservable assumption nearly doubles the payback period. That is why AI ROI is presented as a range with explicit sensitivity rather than a point estimate: a single confident figure invites the steering committee to treat an untested assumption as a commitment, and it hands the project a number it can only lose against.
Establishing Metrics for Measuring the Return
- Express the benefit in a metric the business already measures — indemnity leakage, cost per claim, first-contact resolution, stockout rate, days sales outstanding. A metric invented for the project cannot be audited and will not be credited.
- Capture the baseline before deployment. Baselines reconstructed after go-live are always disputed.
- Name a benefit owner in the business — normally the process owner, not the project manager — and a finance partner who agrees the calculation method in writing before build starts.
- Build in a counterfactual: a holdout group or a champion/challenger split, so the change can be attributed to the model rather than to a seasonal swing or a concurrent process redesign.
- Set a measurement cadence and a named review date at which modeled benefit is compared against realized benefit.
Cost-Benefit Analysis for Stakeholder Decision-Making
The financial spine of the business case is short: the benefit model with its assumptions exposed, TCO over three years, a payback range, sensitivity on the one or two assumptions that actually move the answer, and a recommendation. Show the do-nothing baseline beside it, and keep a stop option visible — a case with an honest downside and a defined exit is more fundable than one without. The wider assembly of the business case, including its non-financial justification, belongs to the later section on supporting business case creation.
The Exam Behaviors to Memorize
- A model that improves accuracy without changing a decision or an action returns zero. A demand forecast that is five points more accurate but never changes a replenishment order has produced nothing.
- Soft savings count only if the hours leave the cost base or are redeployed to revenue. Finance will not book "productivity hours saved" as return.
- Benefits are agreed with finance and the business owner before build, not calculated by the project team afterwards.
- Run-rate cost, not build cost, decides whether a marginal use case is worth operationalizing. If annual benefit is smaller than annual run cost, stop at the gate however good the model is.
- ROI is re-baselined at phase gates exactly as scope is: after Phase 2 you know the real data cost, and after Phase 5 you know the achievable lift.
A data science team reports that a new demand-forecasting model cuts mean absolute percentage error from 14% to 9% and asks the project manager to record the resulting inventory savings in the benefits tracker. What should the project manager do first?
A finance business partner rejects a business case that shows a five-month payback on a USD 310,000 build for a document-classification AI. The project manager reviews the model. Which correction is most likely to explain the rejection?
Before CPMAI Phase 2 begins, an executive sponsor asks the project manager for "one ROI number" for a fraud-detection initiative so it can be ranked against other portfolio investments. How should the project manager present the case?