16.2 Procedural Integrity Monitoring & Competency Assessments

Procedural Integrity Monitoring & Competency Assessments

Exam Tip: The QASP-S exam places significant emphasis on Procedural Integrity (also referred to as Treatment Fidelity). You must be able to calculate integrity percentages using the standard mathematical formula: $\frac{\text{Correct Steps}}{\text{Total Steps}} \times 100$. Understand the difference between Treatment Drift (the technician's implementation drifts from the plan) and Observer Drift (the observer's scoring drifts from the operational definition). When a technician exhibits low integrity, the correct behavioral supervisory response is never immediate termination or punitive disciplinary write-ups; it is supportive BST re-training, protocol clarification, and environmental restructuring.

In Applied Behavior Analysis, the validity of all clinical conclusions rests upon the assumption that the independent variable (the behavioral intervention) was manipulated precisely as described. Procedural integrity—frequently termed treatment fidelity—refers to the extent to which clinical procedures, behavior intervention plans, teaching programs, and environmental accommodations are implemented exactly as written and designed (Gresham, 1989; Peterson et al., 2007). When a client fails to acquire a target skill or exhibits an exacerbation of challenging behavior, clinicians cannot determine whether the intervention itself was fundamentally flawed or whether it was simply implemented with poor fidelity unless objective procedural integrity data have been systematically gathered.

---

The Direct Correlation Between Fidelity and Client Outcomes

Decades of empirical literature (e.g., St. Peter Pipkin et al., 2010; DiGennaro Reed et al., 2007) demonstrate that treatment integrity is not merely a bureaucratic compliance exercise; it is the primary determinant of therapeutic success:

┌─────────────────────────────────────────────────────────────────────────────┐
│                     CLINICAL IMPACT OF PROCEDURAL INTEGRITY                 │
├──────────────────────────┬──────────────────────────────────────────────────┤
│ HIGH FIDELITY (≥ 90%)     │ • Rapid, stable rates of skill acquisition       │
│                          │ • Swift reduction in problem behaviors           │
│                          │ • Prevents accidental intermittent reinforcement │
│                          │ • Empirical validation of treatment efficacy     │
├──────────────────────────┼──────────────────────────────────────────────────┤
│ LOW FIDELITY (< 80%)     │ • Stalled or fluctuating learning curves         │
│                          │ • Extinction bursts turn into strengthened       │
│                          │   problem behavior via intermittent reinforcement│
│                          │ • Erroneous clinical decisions (abandoning an    │
│                          │   effective plan because of flawed delivery)     │
└──────────────────────────┴──────────────────────────────────────────────────┘

The Peril of Intermittent Reinforcement in Behavior Reduction

The most catastrophic consequence of low procedural fidelity occurs during the implementation of extinction protocols or Differential Reinforcement of Alternative Behavior (DRA). Consider an attention-maintained shouting behavior slated for extinction. If a technician ignores the shouting on 9 out of 10 instances (90% integrity) but succumbs to frustration and reprimands the child on the 10th instance, the technician has inadvertently placed the shouting on a variable-interval (VI) or variable-ratio (VR) schedule of intermittent reinforcement. Intermittent reinforcement drastically increases the behavior's resistance to extinction, resulting in higher peak intensities and prolonged behavioral escalation. In this scenario, 90% fidelity is clinically dangerous; behavior reduction protocols demand near 100% adherence to ensure extinction contingencies remain intact.

---

Task-Analyzed Treatment Integrity Checklists & Mathematical Calculation

To measure procedural integrity objectively, the supervisor must deconstruct the intervention protocol into a task analysis of discrete, observable technician behaviors. Broad evaluations such as "technician ran DTT well" are clinically meaningless.

Core Structural Components of a Fidelity Checklist:

1. Antecedent Preparation: Were required stimuli, communication devices, tokens, and data sheets organized prior to initiating the instructional trial?
2. Prompt Execution: Was the discriminative stimulus ($S^D$) presented clearly without extraneous vocal babble? Was the prescribed prompt delay (e.g., 0-second or 4-second) honored? Was the prompt hierarchy strictly followed?
3. Consequence Delivery: Was correct responding reinforced within 2 seconds with the prescribed stimulus? Was the reinforcer magnitude appropriate? Was the error-correction procedure executed accurately following incorrect responses?
4. Data Recording: Was the trial outcome recorded immediately and accurately following the consequence?

The Mathematical Formula for Treatment Integrity:

$\text{Procedural Integrity Percentage} = \left( \frac{\text{Number of Correctly Implemented Steps}}{\text{Total Number of Planned Steps}} \right) \times 100$

Worked Clinical Calculation 1: Discrete Trial Training (DTT) Protocol

A QASP-S evaluates a technician running a 10-trial block of receptive identification. Each trial consists of 5 task-analyzed steps (50 total steps across the block):

1. Gain learner attention.
2. Present $S^D$ once clearly.
3. Wait 3-second response latency.
4. Deliver correct consequence (reinforce or error-correct).
5. Record data and clear table (ITI < 3 seconds).

During the 10-trial observation, the technician:

• Failed to gain attention on 2 trials (Steps 1 & 6 missed).
• Repeated the $S^D$ twice on 1 trial (Step 7 missed).
• Handled all other 47 steps with flawless precision.
• Total correct steps = $50 - 3 = 47$.

$$\text{Integrity Percentage} = \left( \frac{47}{50} \right) \times 100 = 94%$$$Conclusion:* The technician achieved 94% procedural integrity, exceeding the standard 90% instructional mastery threshold.

Worked Clinical Calculation 2: Functional Communication Training (FCT) Protocol

A supervisor observes an ABAT implementing an FCT protocol for escape-maintained task refusal across 15 demand presentations. The protocol specifies 4 discrete steps per presentation (60 total steps):

• Present task demand with visual schedule.
• Immediately prompt the replacement mand ("Break please") within 2 seconds.
• Deliver 30-second break upon mand emission.
• Block physical elopement without verbal reprimand if non-compliance occurs.

The technician correctly executes 42 of the 60 planned steps, frequently giving verbal reprimands ("You need to sit down") and allowing the client to escape without manding on 6 occasions:

$$\text{Integrity Percentage} = \left( \frac{42}{60} \right) \times 100 = 70%$$$Conclusion:* The technician achieved only 70% integrity. This falls well below acceptable clinical thresholds (< 85%), mandating immediate supervisory intervention and structured re-training.

---

Observer Drift vs. Treatment Drift

A critical conceptual distinction tested on the QASP-S examination is the difference between errors made by the clinician delivering the service (Treatment Drift) and errors made by the observer scoring the data (Observer Drift).

┌─────────────────────────────────────────────────────────────────────────────┐
│                     OBSERVER DRIFT VS. TREATMENT DRIFT                      │
├──────────────────────────┬──────────────────────────────────────────────────┤
│ TREATMENT DRIFT          │ OBSERVER DRIFT                                   │
├──────────────────────────┼──────────────────────────────────────────────────┤
│ • The practitioner's in- │ • The observer's/supervisor's measurement of the │
│   vivo implementation of │   behavior shifts away from original operational │
│   the intervention moves │   definitions over time.                         │
│   away from the BIP.     │                                                  │
│ • Examples: Adding       │ • Examples: Observer becomes more lenient or     │
│   unapproved verbal      │   stringent; starts counting minor vocalizations │
│   cues, skipping steps,  │   as aggression; scoring habits shift.           │
│   delaying reinforcement.│                                                  │
│ • Remediated by: BST re- │ • Remediated by: Interobserver Agreement (IOA)   │
│   training, job aids, and│   checks, operational retraining, and booster    │
│   fidelity audits.       │   calibration sessions against video benchmarks. │
└──────────────────────────┴──────────────────────────────────────────────────┘

Clinical Manifestation of Treatment Drift

Treatment drift is an almost universal phenomenon in human services if regular monitoring is absent. Over weeks or months, a technician who was initially trained to mastery might gradually begin delivering informal verbal prompts ("Remember what to do..."), altering the tone of voice, anticipating the child's errors, or shortening the extinction interval. These subtle modifications are rarely malicious; rather, they are shaped by natural contingencies (e.g., helping the child faster reduces immediate session friction). Regular fidelity probes are required to detect and reverse treatment drift before it compromises client learning.

---

Observation Scheduling & In-Situ Probes

How and when procedural integrity is monitored significantly affects data validity:

• Scheduled Fidelity Audits: The technician is aware that the supervisor is arriving at 10:00 AM on Tuesday to score treatment fidelity. While necessary for scheduled mentorship, scheduled audits are highly vulnerable to the Hawthorne Effect (reactivity)—the technician exerts maximum effort and follows the protocol perfectly while the supervisor holds the clipboard, only to revert to drifting habits once the supervisor leaves.
• Unscheduled Fidelity Probes: The supervisor conducts brief, spontaneous in-situ probes (e.g., observing 10 trials unannounced or dropping into an ongoing naturalistic session). Unscheduled probes provide an authentic, high-validity assessment of typical daily clinical implementation.
• Frequency Standards: In evidence-based ABA programs, procedural integrity should be assessed for each technician at least bi-weekly or monthly per client protocol, with at least 10–20% of all instructional sessions sampled for fidelity.

Comparative Matrix of Treatment Fidelity Assessment Formats

Supervisors utilize diverse measurement formats to evaluate procedural integrity, balancing depth of observation against supervisory time and resource expenditures:

---

Establishing Pre-Service Competencies & Minimum Integrity Thresholds

Before a newly hired behavior technician is assigned to work independently with an autistic client, the QASP-S must ensure the completion of a formal Pre-Service Clinical Competency Assessment. Permitting an untrained or partially trained technician to deliver therapy independently is an ethical violation that jeopardizes client well-being and clinical outcomes.

Essential Components of Pre-Service Assessment:

1. Direct Role-Play Evaluation: Demonstration of core behavioral repertoires (e.g., discrete trial delivery, naturalistic incidental teaching, errorless prompting, three-step prompting, mand training, and functional communication training) across simulated client scenarios.
2. Behavior Reduction & De-escalation Mastery: Demonstration of antecedent environmental arrangements, active blocking of aggressive or self-injurious responses, and execution of functional non-exclusionary and exclusionary safety protocols.
3. Data Collection Fluency: Accurate recording of frequency, duration, latency, partial-interval, and trial-by-trial data while actively interacting with a simulated learner.

Non-Punitive Remediation Hierarchy

When an ongoing procedural integrity probe reveals a score below the clinical threshold (e.g., 72%), the supervisor must adhere to a supportive, non-punitive remediation hierarchy:

• Step 1: Environmental & Material Check: Determine if the deficit was caused by missing timers, inaccessible reinforcers, or confusing datasheets.
• Step 2: Protocol Clarification: Review the written task analysis with the technician to ensure operational clarity.
• Step 3: Immediate BST Booster Session: Conduct instruction, modeling, simulated role-play, and immediate descriptive feedback with re-rehearsal.
• Step 4: Re-Probe: Conduct an in-situ probe within 48 hours to confirm restoration of ≥ 90% integrity. Administrative discipline or punitive write-ups are only considered if a technician persistently and willfully refuses to implement corrective feedback following documented, repeated BST remediation.