13.3 Treatment Integrity Monitoring, Fidelity Measurement, and Data-Based Plan Revision

The Science of Treatment Integrity

In applied behavior analysis, the term treatment integrity—often used interchangeably with procedural fidelity—refers to the extent to which an independent variable (the intervention protocol) is implemented exactly as planned, described, and written (Gresham, 1989; Peterson et al., 1982). While behavior analysts place extraordinary emphasis on measuring the dependent variable (client behavior) with high accuracy and interobserver agreement, an intervention cannot be scientifically or ethically evaluated unless the independent variable is measured with equal empirical rigor.

Without systematic measurement of treatment integrity, clinicians risk making grave inferential errors:

• Type I Error (False Positive): Concluding that an intervention produced client improvement when, in reality, the behavior change resulted from extraneous environmental confounds or unprescribed technician behaviors.
• Type II Error (False Negative): Concluding that an intervention was clinically ineffective and abandoning it, when in reality the intervention was never implemented with sufficient fidelity to test its therapeutic validity.

+--------------------------------------------------------------------------+
|                THE INTEGRITY INTERPRETATION PARADOX                      |
+------------------------------------+-------------------------------------+
| HIGH INTEGRITY (>= 90-95%)         | LOW INTEGRITY (Under 80-85%)         |
| + Stagnant Client Progress         | + Stagnant Client Progress          |
+------------------------------------+-------------------------------------+
| INFERENCE: The clinical plan is    | INFERENCE: The intervention has NOT |
| ineffective. Re-evaluate FBA       | been tested. Do NOT modify the      |
| hypotheses & modify parameters     | clinical plan. Retrain staff via    |
| (schedules, MOs, prompts).         | Behavioral Skills Training (BST).   |
+------------------------------------+-------------------------------------+

---

Sources and Threats to Treatment Integrity

Treatment integrity rarely fails due to malice or intentional disobedience; rather, it degrades due to predictable behavioral and organizational variables:

1. Treatment Drift: The gradual, imperceptible migration of implementer behavior away from the original intervention protocol over time. Technicians develop personalized shortcuts, alter prompt hierarchies, or inadvertently reinforce precursor behaviors.
2. Protocol Complexity: Interventions with excessive steps, complex branching logic, or unrealistic timing parameters experience rapid integrity deterioration compared to streamlined, parsimonious protocols.
3. Inadequate Initial Training: Relying exclusively on verbal or written instructions (e.g., handing a technician a 12-page BSP and asking them to sign an acknowledgment form) produces abysmal implementation fidelity.
4. Conflicting Environmental Contingencies: Staff working in chaotic classrooms or homes where natural reinforcement favors quick, non-prescribed responses (e.g., handing an iPad to a screaming child to silence them immediately) will inevitably abandon extinction procedures.
5. Absence of Performance Feedback: Without continuous, graphic, and descriptive performance feedback from supervisors, staff behavior quickly habituates to sub-criterion performance.

---

Quantitative Measurement of Treatment Integrity

Treatment integrity must be operationalized, observed directly, and quantitatively recorded using task-analyzed fidelity checklists. A task-analyzed checklist breaks the Behavior Support Plan down into discrete, observable implementer behaviors across every phase of the contingency:

1. Antecedent Preparation: Was the environment arranged correctly? Was the visual schedule present? Were noncontingent reinforcement (NCR) intervals timed accurately using a vibrating timer?
2. Instructional Delivery ($S^D$): Was the discriminative stimulus delivered clearly, without unnecessary vocal filler, and when the client was actively attending?
3. Prompt Execution: Did the technician wait the designated prompt-delay latency (e.g., 3 seconds) before delivering the prescribed prompt level? Was the prompt hierarchy followed without unprescribed verbal prompting?
4. Reinforcement Delivery: Was the functional reinforcer delivered within the required temporal window (e.g., within 2 seconds)? Was the magnitude and schedule (e.g., FR1) correct?
5. Extinction and Error Correction: When problem behavior occurred, did the technician strictly withhold the maintaining consequence? Was physical blocking executed neutrally without vocal commentary or eye contact?

The Mathematical Calculation of Treatment Integrity

Procedural fidelity is quantified as a percentage of correctly executed opportunities or component steps:

$\text{Treatment Integrity (\%)} = \left( \frac{\text{Number of Correctly Implemented Steps / Opportunities}}{\text{Total Number of Prescribed Steps / Opportunities Observed}} \right) \times 100\%$

For example, if an assistant behavior analyst observes an RBT execute a discrete-trial functional communication protocol consisting of 12 discrete steps across 10 trials (total of 120 observable opportunities), and the technician correctly executes 102 steps:

$\text{Treatment Integrity} = \left( \frac{102}{120} \right) \times 100\% = 85.0\%$

Interobserver Agreement (IOA) on Procedural Fidelity

Just as behavior analysts collect IOA on client behavior, supervisors must periodically conduct concurrent, independent fidelity observations. Two independent observers (e.g., a BCBA and a BCaBA) simultaneously score the technician's implementation checklist. Fidelity IOA ensures that the measurement of the independent variable is reliable and free from supervisory observer drift.

---

The "Integrity-First" Decision-Making Protocol

When reviewing clinical data, a flat, regressing, or highly variable trend line in the client's progress graph presents an immediate clinical puzzle: Is the intervention failing because the behavioral hypothesis is wrong, or is it failing because the intervention is not actually being implemented?

The Integrity-First Decision Rule provides a standardized clinical algorithm:

                      [ Client Progress is Flat or Regressing ]
                                        |
                                        v
                   [ STEP 1: Directly Measure Treatment Integrity ]
                                        |
                   +--------------------+--------------------+
                   |                                         |
                   v                                         v
     [ Integrity is LOW (Under 80-85%) ]           [ Integrity is HIGH (>= 90-95%) ]
                   |                                         |
- DO NOT alter the BSP or FBA hypotheses    - Re-evaluate Functional Hypothesis
- Deliver Behavioral Skills Training (BST)  - Check for emerging secondary functions
- Simplify protocol & remove barriers       - Adjust intervention parameters:
- Re-measure integrity until >= 90%            * Increase reinforcement magnitude
                                               * Alter prompt fading latency
                                               * Enhance motivating operations (AOs)

Protocol A: Action Plan for Low Integrity ($< 80\text{--}85\%$)

If treatment integrity falls below the clinical threshold (typically $80\text{--}85\%$):

1. Maintain Clinical Plan Stability: Under no circumstances should the assistant behavior analyst change the BSP, add punitive consequences, or conclude that the client is "non-responsive."
2. Execute Behavioral Skills Training (BST): The supervisor must immediately deliver evidence-based BST (Miltenberger, 2003) to the implementers:
   • Instruction: Provide a concise vocal and written rationale of the specific steps missed.
   • Modeling: Role-play the exact execution of the protocol with the technician.
   • Rehearsal: Have the technician practice the protocol in simulated and live clinical contexts.
   • Feedback: Deliver immediate, positive, and corrective feedback until the technician demonstrates $\ge 95\%$ fidelity across three consecutive role-play scenarios.
3. Assess Environmental Context and Simplify: If fidelity remains low despite BST, assess contextual barriers. Are materials disorganized? Is the data sheet overly tedious? Simplify the task analysis and eliminate unnecessary steps.

Protocol B: Action Plan for High Integrity ($\ge 90\text{--}95\%$) with Stagnant Client Data

If treatment integrity is verified at or above $90\text{--}95\%$ across multiple sessions and staff, but client data remain flat across 3 to 5 consecutive data points:

1. Validate Functional Hypotheses: The original FBA or FA may have missed a concurrent maintaining variable. Re-assess whether the problem behavior has developed multiple functions (e.g., escape and peer attention).
2. Adjust Operational Parameters:
   • Reinforcer Quality and Potency: Conduct a fresh preference assessment. The reinforcing value of the reinforcer may have waned due to satiation or developmental shifts.
   • Schedule Density: Thinning the reinforcement schedule may have occurred too rapidly, inducing schedule strain. Shift back to an FR1 continuous schedule.
   • Latency of Reinforcement: Ensure the delay between the replacement mand and the reinforcer delivery is instantaneous (< 1 second).
   • Prompt Fading Speed: The prompt delay may be too aggressive (causing failure) or too sluggish (inducing prompt dependency).
   • Motivating Operation Prepotency: Strengthen antecedent abolishing operations (e.g., deliver NCR more densely or reduce initial task demands further).

---

Treatment Integrity Troubleshooting and Decision Tree Matrix

The following matrix outlines the clinical decision tree, linking integrity metrics and client trajectories to required clinical actions and common anti-patterns: