16.2 Critical Literature Evaluation and Biostatistical Interpretation

Critical Literature Evaluation and Biostatistical Interpretation

Evaluating infectious diseases pharmacotherapy literature requires sophisticated biostatistical and epidemiological literacy. Unlike chronic disease states where superiority trials with placebo controls predominate, infectious diseases registration trials and stewardship studies routinely employ non-inferiority designs, quasi-experimental interrupted time series, and complex observational cohorts. Antimicrobial stewardship pharmacists must master these analytical designs, effect measures, diagnostic test characteristics, and methods for mitigating systematic biases.

---

Randomized Controlled Trials: Superiority vs. Non-Inferiority

Randomized controlled trials (RCTs) represent the highest level of primary clinical evidence. In infectious diseases, trial designs diverge fundamentally based on their underlying clinical hypotheses.

                  SUPERIORITY VS. NON-INFERIORITY HYPOTHESES

 Superiority Trial:                           Non-Inferiority Trial:
 H0: P_new - P_control ≤ 0                   H0: P_new - P_control ≤ -Δ
 H1: P_new - P_control > 0                   H1: P_new - P_control > -Δ

              -Δ                 0                +Δ
 ──────────────┼─────────────────┼─────────────────┼──────────────►
               │                 │                 │  (New - Control)
               │                 │                 │
               │    [───●───]    │                 │  Inconclusive
               │                 │                 │
               │           [───●───]               │  Non-Inferior (excludes -Δ)
               │                 │                 │
               │                 │     [───●───]   │  Superior (excludes 0)

The Rationale for Non-Inferiority (NI) Trials in ID

In life-threatening systemic bacterial infections (e.g., nosocomial pneumonia, bacteremia, complicated intra-abdominal infections), utilizing a placebo control is unethical because effective standard-of-care antimicrobial therapy is lifesaving. Consequently, novel antimicrobials are compared against established active comparators. Establishing that a new drug is superior in clinical cure is rarely feasible or expected when the active comparator already cures 80% to 90% of patients. Instead, registration trials aim to demonstrate non-inferiority—confirming that the new agent is not clinically worse than the active comparator by more than a pre-specified clinical margin ($\Delta$, delta), while potentially offering secondary advantages (e.g., activity against resistant strains, reduced nephrotoxicity, improved oral bioavailability, or less frequent dosing).

The Non-Inferiority Margin ($\Delta$) and Confidence Intervals

• Defining the Margin ($\Delta$): The margin is established a priori based on clinical consensus and historical active-comparator trials vs. placebo. The margin must preserve a substantial fraction (typically $\ge 50\%$) of the active comparator's historical treatment effect over placebo. In modern antimicrobial registration trials, $\Delta$ is typically set between 10% and 12.5% (or 0.10 to 0.125).
• Hypothesis Testing: Non-inferiority is demonstrated if the lower bound of the two-sided 95% confidence interval (or one-sided 97.5% CI) for the difference in cure rates ($p_{\text{new}} - p_{\text{comparator}}$) lies entirely to the right of $-\Delta$ (i.e., is greater than $-\Delta$). If the lower bound is $\ge -\Delta$, non-inferiority is established. If the lower bound also exceeds 0, superiority can be declared without alpha penalty.

Analysis Populations: ITT vs. mITT vs. Per-Protocol (PP)

---

Observational and Quasi-Experimental Study Designs

When randomized trials are unfeasible, stewardship researchers rely on observational and quasi-experimental architectures.

Cohort and Case-Control Studies

• Prospective and Retrospective Cohort Studies:
  • Subjects are categorized by exposure status (e.g., receipt of vancomycin AUC-guided dosing vs. trough-guided dosing) and followed forward (or historically) for the development of outcomes (e.g., acute kidney injury, 30-day mortality).
  • Primary Effect Measure: Relative Risk (RR) or Risk Ratio ($RR = \text{Cumulative Incidence}_{\text{exposed}} / \text{Cumulative Incidence}_{\text{unexposed}}$).
• Case-Control Studies:
  • Subjects are selected based on the presence of the disease/outcome ("cases", e.g., patients with Clostridioides difficile infection) and compared to subjects without the outcome ("controls"). Past exposures (e.g., prior fluoroquinolone therapy) are evaluated retrospectively.
  • Primary Effect Measure: Odds Ratio (OR) ($OR = [A \times D] / [B \times C]$).
  • Rare Disease Assumption: When the outcome prevalence is low (< 5% to 10%), the OR approximates the RR ($OR \approx RR$). In high-incidence settings, the OR significantly overstates the relative risk.

Interrupted Time Series (ITS) Analysis: The Stewardship Gold Standard

Antimicrobial stewardship interventions (e.g., implementing prospective audit and feedback, pre-authorization, or a novel clinical pathway) are implemented across entire hospital units, precluding individual patient randomization. Interrupted Time Series (ITS) with segmented regression analysis is the gold-standard quasi-experimental design for evaluating such interventions.

                 INTERRUPTED TIME SERIES SEGMENTED REGRESSION

   Outcome Metric
   (e.g., DOT/1000 PD)
        ▲
        │                 Pre-Intervention Secular Trend (β1)
        │             ───
        │         ───
        │     ───
        │ ───
        │                       Immediate Level Drop (β2)
        │                   ▼   
        │                   ●
        │                    ───
        │                        ───   Post-Intervention Slope Change (β3)
        │                            ───
        └───────────────────┬─────────────────────────────────►
                      Intervention Point                     Time (Months)

Segmented Regression Mathematical Formulation

The segmented linear regression model is expressed as:

$Y_t = \beta_0 + \beta_1 \cdot T_t + \beta_2 \cdot D_t + \beta_3 \cdot P_t + \epsilon_t$

• $Y_t$: The aggregated outcome metric at month $t$ (e.g., Days of Therapy [DOT] per 1,000 Patient-Days, or incidence of MRSA bacteremia per 10,000 bed-days).
• $\beta_0$: Baseline level of the outcome at the start of the pre-intervention period.
• $\beta_1$: Baseline secular trend (pre-intervention slope) per unit time, accounting for whether antibiotic use was already rising or falling prior to the intervention.
• $T_t$: Continuous time variable from the beginning of the observation period.
• $\beta_2$: Immediate level change occurring immediately following the stewardship intervention (step change).
• $D_t$: Indicator dummy variable (0 before intervention, 1 after intervention).
• $\beta_3$: Slope change representing the difference between the post-intervention slope and the pre-intervention slope (longitudinal trajectory change).
• $P_t$: Continuous time variable elapsed since the implementation of the intervention (0 before intervention, $1, 2, 3...$ after intervention).
• $\epsilon_t$: Random error term.

---

Biostatistical Parameters, Hypothesis Testing, and Effect Measures

Hypothesis Testing and Errors

• Null Hypothesis ($H_0$): States there is no true difference between study arms ($p_1 = p_2$).
• Type I Error ($\alpha$): Rejecting the null hypothesis when it is actually true (false positive). Standardly set at $\alpha = 0.05$.
• Type II Error ($\beta$): Failing to reject the null hypothesis when it is actually false (false negative). Standardly set at $\beta = 0.10$ to $0.20$.
• Statistical Power ($1 - \beta$): The probability of detecting a statistically significant difference when a true difference exists (typically 80% to 90%). Underpowered trials risk prematurely dismissing effective therapies.
• P-Value vs. Confidence Interval: A p-value conveys only whether an arbitrary significance threshold was crossed; a 95% Confidence Interval (CI) conveys both statistical significance and the precision of the effect estimate, allowing clinicians to judge clinical significance.

Effect Measures: Risk Reductions, NNT, and NNH

                    2x2 CLINICAL CONTINGENCY MATRIX

                      Outcome Present (+)     Outcome Absent (-)
   Experimental (E)           a                       b          Total E = a + b
   Control (C)                c                       d          Total C = c + d

   Experimental Event Rate (EER) = a / (a + b)
   Control Event Rate (CER)      = c / (c + d)

1. Absolute Risk Reduction (ARR): $ARR = |CER - EER|$
2. Relative Risk Reduction (RRR): $RRR = \frac{|CER - EER|}{CER} = 1 - RR$
3. Number Needed to Treat (NNT): The number of patients who must receive the experimental treatment rather than the control to prevent one additional adverse outcome: $NNT = \frac{1}{ARR}$ Rounding Rule: Always round UP to the nearest whole integer (e.g., $1 / 0.062 = 16.1 \rightarrow \text{NNT} = 17$).
4. Number Needed to Harm (NNH): The number of patients exposed to a therapy for one additional patient to experience a specific adverse event: $NNH = \frac{1}{ARI} = \frac{1}{EER - CER}$ Rounding Rule: Always round DOWN to the nearest whole integer (e.g., $1 / 0.043 = 23.25 \rightarrow \text{NNH} = 23$) to avoid underestimating clinical risk.

---

Diagnostic Test Statistics and ROC Curves

Infectious diseases practice relies heavily on rapid molecular diagnostics, antigen detection assays, and serology.

Sensitivity, Specificity, and Predictive Values

                     DIAGNOSTIC ACCURACY 2x2 TABLE

                          Disease Present (D+)     Disease Absent (D-)
   Test Positive (T+)        True Pos (TP)           False Pos (FP)       PPV = TP/(TP+FP)
   Test Negative (T-)        False Neg (FN)          True Neg (TN)        NPV = TN/(TN+FN)
                           Sensitivity =           Specificity =
                           TP / (TP + FN)          TN / (TN + FP)

• Sensitivity (Sn): Probability that the test is positive in a patient who has the disease ($TP / [TP + FN]$). A test with high sensitivity has few false negatives; useful for screening to rule out disease (SnNOut).
• Specificity (Sp): Probability that the test is negative in a patient who is free of disease ($TN / [TN + FP]$). A test with high specificity has few false positives; useful for confirmation to rule in disease (SpPIn).
• Positive Predictive Value (PPV): Proportion of patients with a positive test who truly have the infection ($TP / [TP + FP]$).
• Negative Predictive Value (NPV): Proportion of patients with a negative test who are truly free of infection ($TN / [TN + FN]$).

Likelihood Ratios (LR+ and LR-)

Likelihood ratios quantify how much a test result shifts the post-test probability of disease, completely independent of population prevalence.

• Positive Likelihood Ratio ($LR+$): Ratio of the probability of a positive test in diseased individuals to that in healthy individuals: $LR+ = \frac{\text{Sensitivity}}{1 - \text{Specificity}} = \frac{\text{True Positive Rate}}{\text{False Positive Rate}}$ Interpretation: An $LR+ > 10$ provides strong, decisive diagnostic evidence to rule in disease.
• Negative Likelihood Ratio ($LR-$): Ratio of the probability of a negative test in diseased individuals to that in healthy individuals: $LR- = \frac{1 - \text{Sensitivity}}{\text{Specificity}} = \frac{\text{False Negative Rate}}{\text{True Negative Rate}}$ Interpretation: An $LR- < 0.1$ provides strong, decisive diagnostic evidence to rule out disease.
• Bayes' Theorem Application: $\text{Pre-Test Odds} \times \text{Likelihood Ratio} = \text{Post-Test Odds}$ $\text{Pre-Test Odds} = \frac{\text{Prevalence}}{1 - \text{Prevalence}}, \quad \text{Post-Test Probability} = \frac{\text{Post-Test Odds}}{\text{Post-Test Odds} + 1}$

Receiver Operating Characteristic (ROC) Curves and AUC

• The ROC curve plots the True Positive Rate (Sensitivity) on the y-axis against the False Positive Rate ($1 - \text{Specificity}$) on the x-axis across continuous diagnostic cut-offs (e.g., serum procalcitonin levels, C-reactive protein concentrations).
• Area Under the Curve (AUC / c-statistic): Measures overall diagnostic discrimination across all possible cut-points.
  • $AUC = 0.5$: No discriminatory value (equivalent to a coin flip).
  • $AUC = 0.7 - 0.8$: Acceptable discrimination.
  • $AUC = 0.8 - 0.9$: Excellent discrimination.
  • $AUC > 0.9$: Outstanding discrimination.

---

Controlling for Confounding and Bias in ID Literature

Observational studies evaluating antimicrobials are vulnerable to profound systematic biases that can distort findings.

                  EPIDEMIOLOGICAL BIASES IN ID RESEARCH

 ┌────────────────────────────────────────────────────────────────────────┐
 │ CONFOUNDING BY INDICATION (CHANNELLING BIAS)                          │
 │ • Sicker patients (septic shock, ICU, multiple comorbidities) receive  │
 │   the newer/broader reserve antibiotic; healthier patients receive SoC.│
 │ • Unadjusted analysis falsely shows higher mortality for new drug.     │
 └───────────────────────────────────┬────────────────────────────────────┘
                                     │ Statistical Mitigation
 ┌───────────────────────────────────▼────────────────────────────────────┐
 │ PROPENSITY SCORE MATCHING (PSM) & IPTW                                 │
 │ • Calculates conditional probability of receiving treatment given      │
 │   baseline covariates; balances treatment and control groups.          │
 └───────────────────────────────────┬────────────────────────────────────┘
                                     │ Survival Time Correction
 ┌───────────────────────────────────▼────────────────────────────────────┐
 │ IMMORTAL TIME BIAS                                                     │
 │ • Time window between culture draw and antibiotic switch (e.g., 48h).  │
 │ • Patients MUST SURVIVE 48h to receive the drug; survivor advantage    │
 │   falsely attributed to drug efficacy unless modeled as time-dependent.│
 └────────────────────────────────────────────────────────────────────────┘

1. Confounding by Indication (Channelling Bias)

Occurs when the clinical indication for prescribing a drug is itself associated with the study outcome. In ID, clinicians preferentially prescribe newer, broader-spectrum reserve antimicrobials (e.g., ceftazidime-avibactam, cefiderocol) to patients with advanced sepsis, multi-organ failure, or prior treatment failures. In crude analyses, the new drug appears associated with higher mortality solely because it was channeled to sicker patients.

• Mitigation: Multivariable logistic regression, multivariable Cox proportional hazards modeling, Propensity Score Matching (PSM), and Inverse Probability of Treatment Weighting (IPTW).

2. Immortal Time Bias

Occurs when there is a time interval during follow-up in which the study outcome (death) cannot occur for one treatment group. For example, comparing patients switched to oral step-down therapy versus those maintained on IV therapy: a patient must survive long enough (e.g., 3–5 days) to become clinically stable and receive the oral agent. If the researcher counts follow-up from hospital admission, the step-down cohort receives "immortal time," introducing an artificial survival bias that drastically inflates the apparent benefit of oral step-down.

• Mitigation: Time-dependent Cox regression modeling, landmark analysis, or nested case-control designs.

3. Selection Bias and Loss to Follow-Up

Systematic differences between participants who complete a study and those who drop out. In HIV clinical trials, differential attrition due to intolerable adverse effects or virologic failure can distort long-term durability comparisons if non-completers are ignored (mitigated by classifying non-completers as treatment failures in "Missing = Failure" ITT analyses).