9.2 Critical Appraisal and Evidence Fit

Appraising evidence for the client in front of you

Part 2 does not usually ask candidates to compute statistics in isolation. It asks whether a psychologist can use research responsibly when the facts are messy. Critical appraisal means evaluating the quality, relevance, and limits of evidence before applying it. The exam may show a treatment with strong support, a client with complicating features, and several plausible next steps. The best answer keeps the evidence base in view while responding to the specific clinical problem.

The first question is methodological: how much confidence should the evidence receive? In evidence hierarchies, systematic reviews and well-controlled randomized trials sit above single trials, which sit above quasi-experimental and correlational designs, case series, expert opinion, and testimonials. Stronger support comes from replicated findings, clinically meaningful (not merely statistically detectable) outcomes, valid measurement, low attrition, and samples that resemble the target client.

Weaker support comes from uncontrolled impressions, very small or unrepresentative samples, surrogate outcomes, and studies that never measured the outcome being promised. Part 2 may not name these designs directly, but it expects the same reasoning.

When evidence is almost ready to use

A common Part 2 pattern is an evidence-based treatment that is almost, but not quite, ready to deploy. A manualized intervention may be appropriate for panic symptoms, yet the stem shows active psychosis, significant cognitive limitation, a language mismatch, or immediate self-harm risk. The strongest answer does not reject science and it does not force the protocol; it sequences care. The psychologist might stabilize risk, obtain consultation, adapt materials, arrange a qualified interpreter, or choose a validated alternative that better fits the presentation.

A second pattern is overclaiming from data. A screening score may indicate the need for further evaluation but does not establish a diagnosis; a positive screen has a known false-positive rate. A test norm valid for one population may be questionable for another. A statistically noticeable change may have a trivial effect size and no impact on daily functioning. Critical appraisal protects the client from decisions that look technical but rest on weak fit.

Use this evidence-fit checklist while studying vignettes:

• Identify the claim being made about an assessment or intervention.
• Ask whether the method measures or changes the actual target problem.
• Check whether the client resembles the population for which the method has support.
• Look for risk factors that change timing or require a higher level of care.
• Consider whether adaptation would preserve the active ingredients of the method.
• Document uncertainty and seek consultation when fit is unclear.

Balancing structure, preference, and standards

Evidence fit is not license to abandon structure. A clinician should not swap a supported trauma intervention for an untested favorite merely because a client is complex. The better move is to use the evidence base as a framework, then adjust language, pacing, examples, measurement, coordination, and safety planning. If the necessary adaptation would alter the active elements so much that the mechanism is lost, the psychologist should select a different evidence-supported option or consult.

Client preference matters but does not override competence, safety, or validity. A client may refuse assessment, a family may request an unsupported technique, or an agency may pressure for a quick conclusion. The scientifically oriented response honors preferences through education and shared decision making while maintaining professional standards: offer choices among appropriate options, explain risks and benefits, and document the rationale. It also helps to recognize the threats to validity the exam embeds in study descriptions. A confounding variable means an alternative explanation was not ruled out.

A small or homogeneous sample limits generalization. The absence of a control or comparison condition means improvement could reflect spontaneous recovery, regression to the mean, or placebo response rather than the treatment. A short follow-up cannot establish durability. Publication and allegiance bias inflate apparent effects. Candidates do not compute these, but they should distrust a claim built on a study that ignores them.

For Part 2, read for constraints. The exam rarely rewards the answer that sounds most enthusiastic about research or the one that dismisses research as too narrow. It rewards disciplined application: identify what the evidence actually says, judge where it fits this client and setting, name where it does not, and choose the responsible step that follows, whether that is to proceed, adapt, sequence, consult, or select an alternative.