6.2 Biostatistics: ARR, RRR, NNT, NNH, Confidence Intervals & P-values

Biostatistics: ARR, RRR, NNT, NNH, Confidence Intervals & P-values

Biostatistics provides the quantitative tools required to interpret therapeutic trial results, separate true pharmacological effects from random variation, and translate research data into actionable patient care decisions. For Canadian pharmacists, evaluating drug promotional materials, manufacturer claims, and clinical guidelines requires fluent mastery of risk metrics, hypothesis testing, error boundaries, and confidence interval interpretation.

---

Quantitative Clinical Trial Metrics: The 2x2 Table

Most binary clinical trial outcomes (such as stroke vs. no stroke, mortality vs. survival, or adverse reaction vs. no reaction) can be organized into a standard $2 \times 2$ contingency table:

From these foundational event rates, clinical biostatisticians derive relative and absolute effect metrics.

---

Relative Risk, Absolute Risk Reduction, and Relative Risk Reduction

1. Relative Risk (Risk Ratio, $RR$)

Relative Risk is the ratio of the probability of an outcome occurring in the experimental group compared to the control group:

$RR = \frac{EER}{CER}$

• Interpretation:
  • $RR = 1.0$: There is no difference in risk between the experimental and control arms (the null value).
  • $RR < 1.0$: The experimental intervention decreases risk (protective effect).
  • $RR > 1.0$: The experimental intervention increases risk (harmful effect).

2. Absolute Risk Reduction ($ARR$)

Absolute Risk Reduction (also termed Risk Difference) represents the absolute arithmetic difference in event rates between the two groups:

$ARR = |CER - EER| = CER - EER \quad (\text{when } CER > EER)$

$ARR$ directly reflects the true clinical impact in the population, answering the question: "Out of 100 patients treated with this therapy, how many will be spared an adverse clinical event?"

3. Relative Risk Reduction ($RRR$)

Relative Risk Reduction represents the proportional reduction in risk achieved by the experimental intervention relative to baseline control risk:

$RRR = 1 - RR = \frac{CER - EER}{CER} = \frac{ARR}{CER}$

Number Needed to Treat (NNT) and Number Needed to Harm (NNH)

Number Needed to Treat ($NNT$)

$NNT$ is the number of patients who must receive a specific therapeutic intervention for a defined period of time for one additional patient to experience a beneficial clinical outcome (or avoid an adverse event) compared to control:

$NNT = \frac{1}{ARR} = \frac{1}{CER - EER}$

(Note: $ARR$ must be entered in decimal form, e.g., $5\% = 0.05$)

Mandatory Rounding Convention for $NNT$:

Always round UP to the next whole integer.

• Mathematical Rationale: If $NNT$ calculates to $23.1$ or $23.8$, treating 23 patients will prevent fewer than 1 full event ($23 \times 0.042 = 0.966$ events prevented). To ensure at least one full event is prevented, 24 patients must be treated. Rounding down would overestimate therapeutic benefit.
• Temporal Requirement: $NNT$ is clinically meaningless without an explicit timeframe. An $NNT$ of 25 over 1 year represents far greater clinical benefit than an $NNT$ of 25 over 10 years.

Absolute Risk Increase ($ARI$) and Number Needed to Harm ($NNH$)

When an intervention increases the incidence of an adverse drug event or toxicity, the difference in event rates is the Absolute Risk Increase ($ARI$):

$ARI = EER - CER \quad (\text{when } EER > CER)$

Number Needed to Harm ($NNH$) indicates the number of patients who must be exposed to the intervention for a defined period before one additional patient experiences the adverse event:

$NNH = \frac{1}{ARI} = \frac{1}{EER - CER}$

Mandatory Rounding Convention for $NNH$:

Always round DOWN to the previous whole integer.

• Clinical Safety Rationale: If $NNH$ calculates to $52.8$, treating 52 patients already exposes the cohort to the toxic threshold. To maintain conservative vigilance and avoid underestimating harm, round down to 52. Rounding up to 53 would falsely suggest that patients can tolerate higher exposure before experiencing toxicity.

Step-by-Step Worked Clinical Biostatistics Problem

Clinical Trial Scenario: A 3-year randomized, double-blind trial investigates a novel SGLT2 inhibitor versus placebo in 5,000 patients with heart failure with reduced ejection fraction ($2,500$ patients per arm).

• Primary Efficacy Outcome (CV Death or HF Hospitalization):
  • Placebo arm: $345$ events out of $2,500$ patients.
  • Treatment arm: $240$ events out of $2,500$ patients.
• Primary Safety Outcome (Symptomatic Mycotic Genital Infections):
  • Treatment arm: $68$ events out of $2,500$ patients.
  • Placebo arm: $20$ events out of $2,500$ patients.

Step-by-Step Efficacy Calculations:

1. Calculate Control and Experimental Event Rates: $CER = \frac{345}{2,500} = 0.138 \quad (13.8\% implication)$ $EER = \frac{240}{2,500} = 0.096 \quad (9.6\% implication)$
2. Calculate Relative Risk ($RR$): $RR = \frac{EER}{CER} = \frac{0.096}{0.138} = 0.6957 \approx 0.70$
3. Calculate Absolute Risk Reduction ($ARR$): $ARR = CER - EER = 0.138 - 0.096 = 0.042 \quad (4.2\%)$
4. Calculate Relative Risk Reduction ($RRR$): $RRR = \frac{ARR}{CER} = \frac{0.042}{0.138} = 0.3043 \quad (30.43\% \approx 30.4\%)$
5. Calculate $NNT$: $NNT = \frac{1}{ARR} = \frac{1}{0.042} = 23.81$
   • Applying the round-up rule: $NNT = 24$ over 3 years.

Step-by-Step Safety Calculations:

1. Calculate Safety Event Rates: $EER_{\text{harm}} = \frac{68}{2,500} = 0.0272 \quad (2.72\%)$ $CER_{\text{harm}} = \frac{20}{2,500} = 0.0080 \quad (0.80\%)$
2. Calculate Absolute Risk Increase ($ARI$): $ARI = EER_{\text{harm}} - CER_{\text{harm}} = 0.0272 - 0.0080 = 0.0192 \quad (1.92\%)$
3. Calculate $NNH$: $NNH = \frac{1}{ARI} = \frac{1}{0.0192} = 52.08$
   • Applying the round-down rule: $NNH = 52$ over 3 years.

Clinical Synthesis: Over 3 years, for every 24 patients treated with the novel drug instead of placebo, 1 cardiovascular death or heart failure hospitalization is prevented ($NNT = 24$), while 1 additional mycotic infection occurs for every 52 patients treated ($NNH = 52$).

---

Odds Ratio (OR) vs. Hazard Ratio (HR)

Odds Ratio ($OR$)

In retrospective case-control studies where total population denominator counts are unavailable, incidence cannot be measured, precluding calculation of Relative Risk. Instead, researchers compute the Odds Ratio ($OR$):

$OR = \frac{\text{Odds of exposure among cases}}{\text{Odds of exposure among controls}} = \frac{a / c}{b / d} = \frac{a \times d}{b \times c}$

Hazard Ratio ($HR$)

A Hazard Ratio is derived from survival (time-to-event) analysis, typically using Cox proportional hazards regression modeling. Unlike Relative Risk—which simply counts whether an event occurred by the end of a trial—the Hazard Ratio compares the instantaneous event rate occurring at any given point in time during follow-up, accounting for:

1. Varying durations of patient follow-up;
2. Censoring (participants who complete the study without an event, withdraw, or are lost to follow-up);
3. The timing of events (detecting whether a drug delays events early even if lifetime cumulative incidence converges).

• Interpretation:
  • $HR = 1.0$: Equivalent instantaneous event hazard in both arms (null value).
  • $HR < 1.0$: Experimental therapy reduces the hazard rate over time (protective).
  • $HR > 1.0$: Experimental therapy accelerates the hazard rate over time (hazardous).

---

Statistical Hypothesis Testing: Type I Error, Type II Error, and Power

Clinical trials utilize inferential statistics to test formal hypotheses regarding whether observed differences reflect true pharmacological reality or mere random sampling fluctuation.

                            THE HYPOTHESIS TESTING MATRIX

                                         TRUE STATE OF REALITY
                                ┌───────────────────────┬───────────────────────┐
                                │   H₀ is TRUE          │   H₀ is FALSE         │
                                │ (No True Difference)  │ (True Difference Exists)│
┌───────────────────────────────┼───────────────────────┼───────────────────────┤
│ REJECT H₀                     │   TYPE I ERROR (α)    │   CORRECT DECISION    │
│ (Conclude Difference Exists)  │   "False Positive"    │   POWER (1 - β)       │
├───────────────────────────────┼───────────────────────┼───────────────────────┤
│ FAIL TO REJECT H₀             │   CORRECT DECISION    │   TYPE II ERROR (β)   │
│ (Conclude No Difference)      │       (1 - α)         │   "False Negative"    │
└───────────────────────────────┴───────────────────────┴───────────────────────┘

1. The Hypotheses

• Null Hypothesis ($H_0$): There is no true difference between the experimental treatment and control ($RR = 1.0$, $ARR = 0$).
• Alternative Hypothesis ($H_1$): A true difference exists between the treatments ($RR \neq 1.0$, $ARR \neq 0$).

2. Type I Error ($\alpha$) — "False Positive"

Occurs when researchers reject the null hypothesis when $H_0$ is actually true, falsely concluding that an ineffective drug is effective. In biomedical research, the alpha threshold is conventionally set at $\alpha = 0.05$ ($5\%$), meaning researchers accept a maximum $5\%$ probability that a positive finding occurred purely by chance.

3. Type II Error ($\beta$) — "False Negative"

Occurs when researchers fail to reject the null hypothesis when $H_0$ is actually false, missing a true therapeutic effect. Beta is conventionally set between $\beta = 0.10$ and $0.20$ ($10\%$ to $20\%$).

4. Statistical Power ($1 - \beta$)

Statistical power is the probability that a trial will detect a statistically significant difference if a true difference of a given magnitude actually exists. Standard clinical trial design demands power of at least $80\%$ ($1 - 0.20$) or $90\%$ ($1 - 0.10$).

• Key Determinants of Statistical Power:
  • Sample Size ($N$): Larger sample size increases power.
  • Effect Size: Detecting a large therapeutic difference requires fewer patients than detecting a subtle difference.
  • Baseline Event Rate: In trials with rare clinical events, total event count—not just patient count—dictates power. If fewer events occur than projected, power plummets.
  • Significance Level ($\alpha$): A more stringent alpha (e.g., $0.01$ vs. $0.05$) reduces power.

5. The $p$-value: Definition and Misconceptions

The $p$-value is the probability of obtaining an effect equal to or more extreme than the observed trial result, assuming that the null hypothesis is true.

• If $p < 0.05$: The result is deemed statistically significant, and $H_0$ is rejected.
• Crucial Clinical Truths:
  • A $p$-value does not indicate the probability that the hypothesis is true.
  • A $p$-value does not reflect the clinical magnitude or importance of the effect. With massive sample sizes ($N > 50,000$), a clinically trivial difference (e.g., systolic blood pressure reduction of $0.4\text{ mmHg}$) can achieve $p < 0.001$.
  • A $p$-value $\ge 0.05$ does not prove that treatments are equivalent; it merely indicates that the study lacked sufficient evidence to reject the null hypothesis ("absence of evidence is not evidence of absence").

---

Confidence Intervals: Interpretation of Difference vs. Ratio Metrics

A 95% Confidence Interval ($95\%\text{ CI}$) represents the range of plausible values within which the true population effect parameter lies with $95\%$ certainty. Confidence intervals convey both statistical significance and the precision of the estimate (narrow intervals reflect high precision; wide intervals indicate low precision due to small sample size or sparse events).

                  HOW TO INTERPRET A 95% CONFIDENCE INTERVAL

 DIFFERENCE METRICS (ARR, Mean Difference)        RATIO METRICS (RR, OR, HR)
 ─────────────────────────────────────────        ──────────────────────────
 Null Value = 0                                   Null Value = 1.0

 CI: [0.02 to 0.08]                               CI: [0.65 to 0.88]
 ────●──────●──────►                              ────●──────●──────►
 0                                                0          1.0
 Excludes 0 ──► STATISTICALLY SIGNIFICANT         Excludes 1.0 ──► STATISTICALLY SIGNIFICANT
               (p < 0.05)                                         (p < 0.05)

 CI: [-0.01 to 0.06]                              CI: [0.82 to 1.15]
 ────●───0──●──────►                              ────●───1.0──●────►
         ▲                                                ▲
 Crosses 0 ──► NOT STATISTICALLY SIGNIFICANT      Crosses 1.0 ──► NOT STATISTICALLY SIGNIFICANT
               (p ≥ 0.05)                                         (p ≥ 0.05)

Difference Metrics vs. Ratio Metrics Summary:

1. Difference Metrics ($ARR$, Absolute Risk Increase, Mean Difference in Blood Pressure or HbA1c):
   • Null Value is $0$.
   • Statistically significant ($p < 0.05$) if the $95\%\text{ CI}$ excludes $0$ (both boundaries are positive or both are negative).
   • Not statistically significant ($p \ge 0.05$) if the $95\%\text{ CI}$ crosses $0$ (lower bound is negative, upper bound is positive).
2. Ratio Metrics ($RR$, Odds Ratio, Hazard Ratio):
   • Null Value is $1.0$.
   • Statistically significant ($p < 0.05$) if the $95\%\text{ CI}$ excludes $1.0$ (both boundaries $< 1.0$ or both $> 1.0$).
   • Not statistically significant ($p \ge 0.05$) if the $95\%\text{ CI}$ crosses $1.0$ (e.g., $0.85$ to $1.20$).

Clinical Significance vs. Statistical Significance

A statistically significant finding ($p < 0.05$, CI excluding null) does not guarantee that the intervention is clinically meaningful. Clinicians must compare the confidence interval bounds against the Minimal Clinically Important Difference (MCID)—the smallest change in outcome that patients or clinicians perceive as beneficial. If a confidence interval falls entirely below the MCID, the drug's effect is clinically trivial despite statistical significance.