Procedural Integrity and Effectiveness Decisions

Two Data Streams, One Decision

Procedural integrity (also called treatment integrity or fidelity) is the degree to which an intervention is implemented as written. Effectiveness is the degree to which the client's behavior changes in the intended direction. Domain H decision items almost always require reading both before changing anything. Acting on outcome data alone is the single most common error the exam punishes.

Integrity is typically measured as the percentage of plan steps performed correctly, collected via direct observation, permanent-product review, or self-report (least reliable). It is reported per implementer and over time so the BCBA can see drift.

The logic is causal. An intervention's outcome data are only interpretable conditional on integrity. If you do not know whether the plan was run correctly, a flat graph tells you nothing about the procedure — it could be a bad procedure or a good procedure run badly. This is why integrity is a prerequisite for any effectiveness conclusion.

The 2x2 Decision Matrix

Cross the two streams and the correct action becomes clear. Memorize this matrix; the exam tests every cell.

The two diagonal traps are where candidates lose points:

• Improving + Low integrity looks like success but is unearned — the gains may be due to uncontrolled variables, and the plan as written has not actually been tested. You cannot yet claim the procedure works.
• Flat + Low integrity is not evidence that the plan is wrong. The plan has not had a fair test. Fixing integrity, not redesigning the intervention, is the first move.

Only the Flat + High integrity cell licenses a real change to the procedure, because only then has the written plan been adequately tested and found wanting.

Diagnosing Low Integrity vs. a Wrong Plan

When integrity is low, the fix targets the implementation system, not the procedure: improve staff training, provide clearer materials and prompts, deliver performance feedback, simplify steps, or improve contextual fit so the plan is runnable in the real setting. Performance feedback and BST (behavioral skills training) are the workhorses here. Do not abandon a sound, function-based plan before it has been tested under adequate implementation.

When integrity is high but behavior is not improving, the plan itself is suspect. Systematically review:

• Is the assessment hypothesis (function) still correct, or has function shifted?
• Does the reinforcer still have value (satiation, competing reinforcers)?
• Is the prompting/fading appropriate, and is the schedule dense enough?
• Is the mastery criterion realistic, and is the measurement system valid (right dimension, reliable definitions)?
• Does the plan still have social validity for the client and stakeholders?

Worked example. A DRA plan shows a flat graph. Integrity data reveal the aide is delivering the reinforcer only on about 55% of correct mands. This is a low-integrity case. The BCBA does not conclude DRA failed; she runs a feedback and retraining session, provides a one-page prompt, and re-checks integrity. Once integrity reaches 95% and the graph is still flat for several sessions, now she has license to modify the procedure — perhaps the reinforcer has lost value and a new preference assessment is needed.

Reading the Graph Like an Analyst

A data-based decision is not a reaction to one point. Use the visual-analysis dimensions you learned for single-case data: level (mean), trend (slope/direction), variability (bounce), and immediacy/latency of change after a phase shift. Layer in risk (is dangerous behavior escalating?) and social validity (do stakeholders still find the goals, procedures, and outcomes acceptable?).

Two exam rules follow directly. Never choose "continue unchanged" merely because the written plan looks good on paper — the data and integrity must support continuation. And never choose "change everything" without first checking integrity, measurement validity, and risk — wholesale change discards information and may scrap a plan that simply was not implemented.

The credited answer is almost always a specific, data-justified next step: strengthen integrity, re-densify reinforcement, reassess the reinforcer, revise the goal, or begin generalization. The skill the exam is measuring is your ability to name the one move the data point to — and to refuse to act on outcome data divorced from integrity.

It also helps to know why integrity drifts, because the fix depends on the cause. Common causes are insufficient training (the implementer never learned the step), performance deficits (they can do it but do not — a motivation/feedback problem), plan complexity (too many steps to run reliably), and poor contextual fit (the plan cannot be run given staffing or schedule). Training fixes the first, feedback the second, simplification the third, and re-design the fourth — so diagnosing the cause precedes picking the remedy.

A last caution: protect measurement validity before trusting any decision. If observers are not scoring the operational definition consistently — low interobserver agreement (IOA) — a flat or noisy graph may reflect bad data rather than bad treatment. So when outcomes look wrong, the analyst checks not only integrity but also whether the measurement system itself is sound. Decisions built on invalid data are indefensible, and the exam rewards candidates who verify the data before changing the plan.