Critiquing Graphs and Choosing Defensible Designs
Key Takeaways
- Visual analysis should consider level, trend, variability, immediacy, overlap, and consistency together.
- Design choice depends on the research question, target behavior, ethics, reversibility, and context.
- Weak graphs often have unstable baseline, simultaneous changes across tiers, high overlap, or confounded condition changes.
- A defensible design balances experimental rigor with client safety, social significance, and feasibility.
Critiquing graphs like an examiner
Start with the question the graph is supposed to answer. A graph comparing two treatments should show clear separation between conditions. A graph testing a treatment package should show replicated change after intervention. A graph shaping performance should show behavior tracking each criterion.
Use visual analysis features together. Level is the average or typical value within a phase. Trend is the direction of data. Variability is bounce within a phase. Immediacy is how quickly behavior changes after a condition change. Overlap is how much data share the same range across phases. Consistency asks whether similar phases show similar patterns.
| Problem in graph | Why it matters | Better move |
|---|---|---|
| Baseline already improving | Treatment effect is ambiguous | Extend baseline if ethical or use stronger replication |
| All tiers change at once | Staggered control is lost | Delay intervention across tiers |
| High overlap across conditions | Effects are unclear | Adjust procedure or collect more data |
| No integrity data | IV may not be implemented | Add procedural integrity measurement |
| Confounded phase change | More than one variable changed | Change one key variable at a time |
Design selection is not a memorization task. Choose reversal for reversible behavior and ethical withdrawal. Choose multiple baseline for durable skill acquisition or unsafe withdrawal. Choose multielement for fast comparisons when conditions can be discriminated. Choose changing criterion for gradual performance goals.
Defensible design checklist
- The dependent variable directly represents the problem or goal.
- The independent variable is operationalized and feasible.
- The design includes prediction, verification, and replication.
- Alternative explanations are anticipated and reduced.
- The design is ethical for the client and context.
A graph can be clinically useful even when experimental control is weak. For exam purposes, do not overclaim. Say the data suggest improvement when the design lacks replication; reserve functional relation language for clear, replicated control.
A multiple-baseline graph shows all three participants improving before intervention is introduced to the second and third participants. What is the best critique?
A BCBA wants to evaluate a toilet-training intervention after continence is acquired. Withdrawing the intervention would likely be unethical and behavior may not reverse. Which design is most defensible?
A graph shows high overlap between baseline and treatment, delayed change after treatment begins, and no replication. What should the analyst conclude?