Verify Data Quality for the Data Preparation Go/No-Go

Two Data Gates, Two Different Questions

CPMAI puts a decision point at the end of each phase, and Domain IV, Task 5 is easy to confuse with the one before it. Keep them apart.

• Domain III, Task 8 closes Phase 2, Data Understanding. It asks: do we have the right raw data? Enough volume, enough history, the right granularity, legally usable, actually obtainable.
• Domain IV, Task 5 closes Phase 3, Data Preparation. It asks: did preparation produce a training-ready dataset that is correct, representative, and free of leakage?

A project can pass the first and fail the second. The raw claims history can be complete and lawful, and the prepared table can still be ruined by a join that multiplied rows, an imputation default nobody sanctioned, or a split that lets the model see the future. This gate exists because the cheapest place to catch that is before a single model is trained.

The Final Quality Assessment Runs on the Prepared Dataset

Re-run quality checks against the table that will actually be fed to the algorithm, not the raw extract you assessed in Phase 2:

• Row and column counts against what the preparation specification predicted.
• The record funnel — how many rows survived each pipeline stage. A pipeline that silently discards 40 percent of the input is a finding, not a footnote.
• Null rate per feature after imputation, against the threshold agreed for each field. A feature that is 60 percent imputed is a feature carrying almost no information.
• Value ranges and referential integrity after joins, plus duplicate key detection.
• Label distribution against the event rate the business believes is real.

Validating That the Transformations Did What Was Intended

Three checks earn their keep here.

Trace records back to source. Pull 20 to 30 prepared rows and walk each field back to the originating system with the data SME present. This unglamorous exercise catches the errors automated checks miss: currency in dollars where the pipeline assumed cents, timestamps in local time where downstream logic assumed UTC, and joins that quietly duplicated rows so one claim now appears four times.

Compare distributions before and after. Imputing a median into 30 percent of a field collapses its variance and can make a genuinely predictive feature look inert. A simple before-and-after comparison per feature exposes it.

Verify the splits. Confirm proportions match the plan, then check overlap at key level — the same customer, claim, or device appearing in both training and test inflates every number you are about to report. Near-duplicates count. For any time-dependent problem, confirm the test period sits strictly after the training period, and that no feature is computed using information dated after the prediction point. Temporal leakage produces a model that looks outstanding at the gate and collapses in production, which is precisely why the check belongs here rather than at model evaluation.

Representativeness and Bias, Re-Checked on the Prepared Data

Bias assessment in Phase 2 tested the raw population. This gate tests the population that survived preparation, because preparation itself can introduce bias the raw data never had.

Dropping incomplete records is the usual mechanism. Incompleteness is not random: it concentrates in whoever the organization captures least well — newer customers, a channel with a weaker feed, a rural branch, a partner-sourced segment, applicants outside the standard employment model. Drop them all and you have built a dataset for a population that is not the population you will serve.

So profile the exclusions. Compare the segment composition of dropped rows against retained rows, and compare the prepared dataset against the real deployment population. When dropped rows correlate with a protected or business-critical segment, that is a bias finding requiring remediation, not routine cleaning. Remediation options, in rough order of preference: recover the missing data at source, impute with an explicit "unknown" indicator rather than dropping, re-weight the sample, or narrow the deployment scope and document the exclusion openly.

The Readiness Decision

You run the gate. The data science lead, the data SME, and the data owner or steward sign; privacy or compliance signs whenever personal data is involved; the sponsor is informed of the outcome and of anything that changes scope. The gate then lands somewhere, and only one of the landing places is a plain go to Phase 4, Model Development.

It can land on a conditional go, which releases a defined subset or a narrowed scope with the limitation written down and a named remediation owner and date against it. It can send the work back to iterate — to Phase 3 to redo transformations, or further back to Phase 2 to acquire more or better data. And it can stop, when preparation has demonstrated that the data cannot support the solution at all.

Returning to an earlier phase is a designed CPMAI outcome, not a project failure. It costs a fraction of what the same discovery costs after the build.

Documenting Findings so the Modeling Team Inherits Them

The deliverable is a data readiness memo (or data quality report) that travels with the dataset into Phase 4. It names the dataset version identifier, the checks run and their results, the imputation and exclusion rules applied, the split definition and why it was chosen, the segments known to be under-represented, every open remediation item with an owner and date, and the decision itself with signatures.

Its purpose is transfer. The modeling team should inherit known limitations, not rediscover them at model evaluation after weeks of tuning, and the operations team should not learn about an excluded segment from a customer complaint. A gate that produces a decision but no memo has done half the job.