Determine If the Data Meets Solution Needs: The Phase 2 Go/No-Go
Key Takeaways
- Task 8 closes CPMAI Phase 2 and controls entry to Phase 3, Data Preparation, where project costs rise sharply.
- Compare found data against the Task 1 requirements item by item, marking each met, partially met, or not met; aggregate judgments hide fatal gaps such as an absent target variable.
- Sufficiency is four questions: volume relative to problem complexity, label availability and quality, coverage of rare but important cases, and history long enough to span the relevant business cycles.
- Four gate outcomes are legitimate — proceed, proceed with a narrowed scope, pause to remediate, or stop — and the decision is recorded in a dated go/no-go memo.
- Stopping at the Phase 2 gate is a cheap success; discovering the same shortfall in Phase 5 costs the entire build.
A Gate, Not a Status Update
Task 8 of Domain III, determine if data meets solution needs, is one of the most consequential decision points in the whole CPMAI methodology. It closes Phase 2, Data Understanding, and controls entry to Phase 3, Data Preparation, where costs rise sharply. Exam items test whether you treat it as a formal gate with named decision-makers and legitimate outcomes, or as a checkpoint everyone walks through on the way to modeling.
Compare Available Data Against Defined Requirements
Start with the data requirements defined in Domain III Task 1 and walk them item by item, not in aggregate. For each required element record what was specified, what was found, and the delta: field present or absent, history depth required versus available, granularity required versus available, refresh frequency required versus available, label availability, and quality against the thresholds set at the time. Mark each item met, partially met, or not met. Aggregate judgments hide fatal gaps: a dataset can satisfy nineteen of twenty requirements and still be unusable because the twentieth was the target variable.
Assess Data Sufficiency for Training Robust Models
Sufficiency is four separate questions, and volume is only the first.
- Volume relative to problem complexity. Two million rows is plentiful for a two-class churn model and thin for a model distinguishing 300 clinical codes. Ask for volume per class or per category, not in total.
- Label availability and quality. Supervised learning needs recorded outcomes. Ask who labeled them, how consistently, and whether two experts would agree. Labels produced as a by-product of an existing process are usually biased toward the cases that process already caught.
- Coverage of rare but important cases. Forty documented pump failures may be all the history there is, and it may not be enough to learn from. The rare class is usually the whole point of the project.
- Enough history to span the relevant cycles. A demand forecast built on nine months has never seen a peak season it will be judged on; a credit model needs enough elapsed time for loans to have defaulted or matured.
Identify Data Gaps and Develop Strategies for Addressing Deficiencies
Every gap has options, and every option has a cost the decision-makers must see.
| Gate finding | Option | Trade-off | Typical decision |
|---|---|---|---|
| Volume adequate, labels missing on 70% of records | Relabel a stratified sample with SMEs | Cost and elapsed time; depends on SME availability | Proceed after a time-boxed labeling sprint, re-gated on completion |
| Too few rare but critical events | Buy external data or pool with a partner | License cost, distribution mismatch, new permission work | Pause; clear the new source under Task 6 before it is used |
| Class of interest at 0.2% prevalence | Generate synthetic or augmented examples | Synthetic data encodes its generator's assumptions; regulators may challenge it | Proceed only with a documented method and a real-data holdout |
| One segment badly under-represented | Narrow scope to the represented segments for release 1 | Lower forecast benefit; expectations to reset with the sponsor | Proceed with narrowed scope, excluded segments documented |
| Target variable never recorded anywhere | Change the target to a recorded proxy | The proxy may not carry the business meaning, and can import bias | Return to Phase 1 and re-frame the problem with the sponsor |
| History does not span a full business cycle | Wait and accumulate data | Delay, and possible loss of the commercial window | Pause to remediate with an agreed re-gate date |
| All material requirements met | Proceed to Phase 3, Data Preparation | Residual risks remain and belong on the risk register | Proceed with documented conditions |
Validate Data Representativeness for Target Use Cases
Sufficiency asks whether there is enough data; representativeness asks whether it is the right data. Compare the dataset's composition against the population the solution will actually serve — geography, channel, product mix, customer size, language, device, time of day — and against the operating conditions at deployment. A claims model trained only on adjudicated claims never sees the ones withdrawn before adjudication. A chatbot trained on written support tickets will meet spoken transcription errors it has never seen. Representativeness gaps produce a model that performs beautifully in evaluation and fails for a specific group in production, which is also where Domain I bias findings originate.
Make the Go/No-Go Decision
Run the gate properly. The decision body is the sponsor or steering committee, with the data owner and the privacy or legal sign-off from Task 6 present; the project manager convenes it and presents, but does not decide alone. The evidence pack is the requirements comparison, the EDA from Task 7, the sufficiency and representativeness assessment, the gap options with costs and durations, the compliance clearance status, and a written recommendation. Four outcomes are legitimate:
- Proceed to Phase 3 as scoped.
- Proceed with a narrowed scope — fewer segments, a simpler target, a smaller persona.
- Pause to remediate the data, with a defined remediation plan and a re-gate date.
- Stop, and return the remaining budget.
Record the decision, the evidence, the dissent, and the conditions in a dated go/no-go memo. It is the artifact Domain I Task 5 expects in the audit trail, and the reference point if the project is questioned later.
The Trap: A Fixed Deadline Is Not Evidence of Data Readiness
The exam's favorite failure here is proceeding to modeling with known-insufficient data because the date is committed. The data does not improve in Phase 4, the shortfall resurfaces in Phase 5 as unexplainable performance, and by then the build is spent. Stopping at the Phase 2 gate is a cheap, successful outcome — the methodology worked, and the organization learned what it must collect before trying again. Continuing with a documented gap and no plan is the one answer that is always wrong.
A claims triage project defined a data requirement of 24 months of adjudicated outcomes carrying adjuster reason codes. Phase 2 found 24 months of claims, but reason codes are populated on only 30% of them. The go-live date is fixed and publicly committed. What should the project manager do?
A retail demand-forecasting project has nine months of clean, complete transaction history at daily granularity across all stores. Which sufficiency concern should the project manager raise most strongly at the Phase 2 gate?
A collections-prioritization model is intended for national release, but 82% of the available training records come from two metropolitan regions where pricing and product mix differ from the rest of the country. What should the project manager recommend at the go/no-go gate?