1.7 Evidence-Based Practice and Research Methodologies

Evidence-Based Practice and Research Methodologies

Evidence-Based Practice (EBP) is the conscientious, explicit, and judicious integration of the best research evidence with clinical expertise and patient values and preferences. For the Psychiatric-Mental Health Nurse Practitioner (PMHNP), competence in research design, statistical interpretation, and critical appraisal is essential. ANCC examination questions test your ability to evaluate clinical literature, interpret epidemiological risk metrics, calculate therapeutic effect sizes, and translate clinical practice guidelines (CPGs) into safe patient care.

The Hierarchy of Evidence

Clinical evidence is structured hierarchically based on study design rigor and susceptibility to bias. PMHNPs must prioritize higher-level evidence when formulating clinical protocols.

                    /\  Level I: Systematic Reviews & Meta-Analyses of RCTs
                   /  \----------------------------------------------------
                  /    \ Level II: Individual Randomized Controlled Trials (RCTs)
                 /      \----------------------------------------------------
                /        \ Level III: Controlled Trials Without Randomization
               /          \----------------------------------------------------
              /            \ Level IV: Case-Control and Cohort Studies
             /              \----------------------------------------------------
            /                \ Level V: Systematic Reviews of Descriptive Studies
           /                  \----------------------------------------------------
          /                    \ Level VI: Single Descriptive or Qualitative Studies
         /                      \----------------------------------------------------
        /                        \ Level VII: Expert Opinion, Guidelines, Case Reports
       ----------------------------

Detailed Breakdown of Levels:

1. Level I (Highest): Systematic Reviews and Meta-Analyses of Randomized Controlled Trials (RCTs). Meta-analyses pool quantitative data across multiple RCTs to increase sample size, enhance statistical power, and produce a definitive effect size.
2. Level II: At least one properly designed, double-blind, randomized controlled trial. RCTs minimize selection bias through random assignment and control for confounding variables.
3. Level III: Well-designed controlled trials without randomization (quasi-experimental studies, non-randomized pre-post designs).
4. Level IV: Well-designed observational studies: Cohort Studies (longitudinal, tracking exposed vs. unexposed groups forward over time for incidence) and Case-Control Studies (retrospective, comparing individuals with a condition to controls without, calculating Odds Ratios).
5. Level V: Systematic reviews of descriptive, qualitative, or epidemiological studies.
6. Level VI: A single descriptive study, qualitative inquiry (e.g., phenomenology, grounded theory), or quality improvement project.
7. Level VII (Lowest): Expert opinion of prominent authorities, clinical consensus panel reports, or single case reports.

---

Quantitative Research & Statistical Principles

Statistical Significance vs. Clinical Significance

• Statistical Significance: Measured by the p-value. A value of $p < 0.05$ indicates a less than 5% probability that the observed difference between treatment groups occurred purely by random chance (rejecting the null hypothesis). However, statistical significance is heavily influenced by sample size ($N$). In large trials, a 1-point reduction on the Hamilton Depression Rating Scale (HAM-D) may reach $p < 0.001$, but carry no meaningful functional benefit.
• Clinical Significance: Refers to whether an intervention produces a noticeable, meaningful improvement in a patient's symptoms, daily functioning, or quality of life. Clinical significance is evaluated using Effect Size measures (e.g., Cohen's $d$) and clinical response/remission thresholds.

Effect Size & Confidence Intervals

• Cohen’s $d$: Quantifies the magnitude of difference between two means in standard deviation units.
  • $d = 0.2$: Small effect
  • $d = 0.5$: Medium effect
  • $d = 0.8$: Large effect
• 95% Confidence Interval (CI): Represents the range of values within which the true population parameter is expected to fall 95% of the time.
  • Critical Exam Rule: If a 95% CI for a ratio metric (Odds Ratio or Relative Risk) includes 1.0, the result is NOT statistically significant (e.g., OR = 1.4, 95% CI [0.95, 2.10] is non-significant).
  • If a 95% CI for a difference in means includes 0, the result is NOT statistically significant.

Type I and Type II Errors

• Type I Error ($\alpha$): A false positive. Rejecting the null hypothesis when it is actually true (claiming a drug works when it does not). Alpha is typically set at 0.05.
• Type II Error ($eta$): A false negative. Failing to reject the null hypothesis when it is actually false (missing a true drug effect). Statistical Power is defined as $1 - eta$; acceptable power in clinical research is $\ge 0.80$ (80%).

---

Key Epidemiological Metrics

Epidemiology measures how disease is distributed across populations. PMHNPs must distinguish between incidence and prevalence.

$\text{Incidence Rate} = \frac{\text{Number of NEW cases occurring in a specific time period}}{\text{Population at risk during that time period}} \times k$

$\text{Prevalence Rate} = \frac{\text{TOTAL number of existing (new + old) cases at a given time}}{\text{Total population at that time}} \times k$

• Incidence: Reflects the risk of contracting a disease. Useful for evaluating the efficacy of primary prevention programs.
• Prevalence: Reflects the overall burden of disease on the healthcare system. Factors that prolong chronic illness without curing it (e.g., effective maintenance antipsychotic therapy) increase prevalence while keeping incidence constant.

---

Therapeutic Metrics: Calculating NNT and NNH

Evaluating the clinical utility of psychiatric medications requires understanding Number Needed to Treat (NNT) and Number Needed to Harm (NNH).

Absolute Risk Reduction (ARR) & NNT

Absolute Risk Reduction is the difference in event rates between the control group ($CER$) and the experimental group ($EER$).

$ARR = CER - EER$

$NNT = \frac{1}{ARR} = \frac{1}{CER - EER}$

• Interpretation: NNT is the number of patients that must receive a specific treatment for one patient to achieve the desired outcome (e.g., symptom remission) who would not have achieved it otherwise.
• Target Value: Lower NNT values indicate higher clinical efficacy. An NNT $< 5$ is considered clinically robust in psychopharmacology.

Absolute Risk Increase (ARI) & NNH

Absolute Risk Increase is the excess risk of an adverse event in the treatment group compared to control.

$ARI = EER_{\text{adverse}} - CER_{\text{adverse}}$

$NNH = \frac{1}{ARI} = \frac{1}{EER_{\text{adverse}} - CER_{\text{adverse}}}$

• Interpretation: NNH is the number of patients treated for one patient to experience a specific adverse event.
• Target Value: Higher NNH values indicate a safer medication. An NNH of 100 means only 1 out of 100 treated patients experiences the side effect.

Clinical Calculation Example

Trial Data: In a 12-week placebo-controlled trial of a novel atypical antipsychotic for acute bipolar mania:

• Mania Remission Rate: Drug Group = 50% ($0.50$), Placebo Group = 30% ($0.30$).
• Severe Weight Gain Rate (>7% body weight): Drug Group = 15% ($0.15$), Placebo Group = 5% ($0.05$).

Calculations:

1. $ARR = 0.50 - 0.30 = 0.20$ (20% reduction)
2. $NNT = \frac{1}{0.20} = 5$
3. $ARI = 0.15 - 0.05 = 0.10$ (10% increase)
4. $NNH = \frac{1}{0.10} = 10$

Clinical Interpretation: The PMHNP must treat 5 patients with the drug for 1 patient to achieve mania remission who would not have remitted on placebo. However, for every 10 patients treated, 1 patient will develop severe weight gain directly due to the drug.