14.1 Study Designs, Bias & Validity

14.1 Study Designs, Bias & Validity

Observational vs. Experimental Study Designs

Epidemiologic study designs are classified into observational and experimental categories based on whether the investigator assigns the intervention. In observational studies, researchers observe natural exposures and health outcomes without actively allocating treatments. In experimental studies, most notably randomized controlled trials (RCTs), the investigator assigns exposures or interventions using controlled randomization.

Cross-Sectional Studies

A cross-sectional study evaluates exposure and disease status simultaneously in a specified population at a single point in time. It measures disease prevalence ("snapshot in time") rather than incidence. Because exposure and outcome are measured concurrently, cross-sectional designs cannot establish temporal precedence or causality. They are efficient for generating hypotheses, estimating public health burdens, and identifying associations (expressed via prevalence odds ratios).

Case-Control Studies

A case-control study is an observational, retrospective design that selects participants based on outcome status: individuals with the disease (cases) are compared to individuals without the disease (controls). Investigators then look backward in time to quantify past exposure frequencies.

• Measure of Association: The primary metric is the Odds Ratio (OR), defined as $OR = \frac{a \cdot d}{b \cdot c}$, where $a$ is exposed cases, $b$ is exposed controls, $c$ is unexposed cases, and $d$ is unexposed controls.
• Key Applications: Ideal for studying rare diseases (e.g., mesothelioma, angiosarcoma) or diseases with long latency periods.
• Major Limitations: Highly susceptible to selection bias and recall bias. Relative risk cannot be calculated directly because population incidence is unknown.

Cohort Studies

A cohort study selects participants based on exposure status: an exposed group and an unexposed group, both initially free of the outcome of interest, are followed over time to compare the incidence of disease. Cohort studies may be prospective (tracking subjects forward from the present) or retrospective (using historical medical records to define past exposure and follow subjects to documented outcomes).

• Measure of Association: The primary metric is Relative Risk (RR) or Risk Ratio, calculated as $RR = \frac{a / (a + b)}{c / (c + d)}$, comparing disease incidence in the exposed ($a/[a+b]$) to incidence in the unexposed ($c/[c+d]$).
• Key Applications: Optimal for studying rare exposures (e.g., occupational chemical exposures, radiation) and establishing temporal sequences.
• Major Limitations: Expense, long duration, and vulnerability to loss to follow-up (attrition bias).

Randomized Controlled Trials (RCTs)

An RCT is the gold standard experimental design for evaluating therapeutic efficacy. Participants are randomly allocated to an intervention group (e.g., novel drug) or control group (e.g., placebo or standard of care). Randomization minimizes baseline confounding by balancing both known and unknown prognostic factors across groups. Blinding (single, double, or triple) prevents investigator and participant bias. RCT results should be analyzed using an intention-to-treat (ITT) protocol, preserving randomization benefits by analyzing participants in their assigned groups regardless of compliance or dropouts.

---

Validity: Internal vs. External

Internal Validity

Internal validity reflects the degree to which a study's observed conclusions represent the true causal relationship within the specific sample studied, free from methodological flaws, bias, or unmeasured confounding. High internal validity requires rigorous study design, appropriate statistical controls, valid measurement instruments, and effective blinding.

External Validity (Generalizability)

External validity denotes the extent to which trial findings can be generalized to broader target populations in real-world clinical practice. A study conducted exclusively in young, healthy male military recruits may exhibit outstanding internal validity but poor external validity when applied to elderly female patients with multiple comorbidities.

---

Major Types of Bias & Mitigating Strategies

Selection Bias

Selection bias occurs when the sample chosen does not accurately represent the target population due to non-random selection or differential retention.

• Berkson Bias: Occurs when hospitalized patients are chosen as controls in case-control studies. Hospitalized controls typically have higher rates of comorbid conditions than the general community, artificially distorting exposure-disease associations.
• Neyman (Prevalence-Incidence) Bias: Occurs when exposure selection excludes patients who die rapidly or recover quickly before study enrollment, over-representing chronic, indolent cases.
• Attrition Bias: Differential loss to follow-up between exposed and unexposed groups in cohort studies.
• Mitigation: Utilize random sampling, consecutive recruitment, standardized enrollment criteria, and maintain follow-up rates $> 80%$.

Information and Measurement Bias

Information bias arises from systemic errors in gathering, recording, or interpreting data.

• Recall Bias: Subjects with a negative outcome (cases) report past exposures more thoroughly or accurately than healthy controls due to search for explanations.
• Observer / Hawthorne Effect: Behavior of study participants changes simply because they know they are being monitored.
• Mitigation: Implement objective data collection methods, structured questionnaires, and double-blinding.

Confounding Bias

Confounding occurs when an unmeasured or uncontrolled third variable is independently associated with both the exposure and the outcome, without being an intermediate step in the causal pathway. For instance, an apparent association between coffee consumption and pancreatic cancer is confounded by smoking, because smoking is correlated with coffee drinking and directly causes pancreatic cancer.

       Confounder (e.g., Smoking)
        /                     \
       v                       v
Exposure (Coffee) ------> Outcome (Pancreatic Cancer)
                       ?

• Mitigation Strategies:
  1. Design Phase: Randomization (balances unknown confounders), Restriction (limiting inclusion to specific sub-groups), and Matching (matching cases and controls by age, sex, or smoking status).
  2. Analysis Phase: Stratification (analyzing exposure-outcome relationships across strata of the confounder) and Multivariable Regression Analysis (logistic or Cox regression adjusting for potential confounders).

Time-Related Biases in Screening

• Lead-Time Bias: Early detection of a disease by screening creates the false appearance of prolonged survival, simply because the diagnosis was established earlier in the disease course, without altering the actual date of death or overall mortality.
• Length-Time Bias: Screening selectively detects slowly progressive, indolent disease cases with long preclinical phases (better prognosis) while missing rapidly progressive, highly aggressive cases that become symptomatic between screening intervals.

---

Algorithmic Evaluation of Study Design & Bias

When analyzing clinical research vignettes on Step 2 CK, utilize the following decision algorithm:

1. Did the investigator assign the exposure or intervention?

   • Yes $\rightarrow$ Experimental Design (RCT). Check for blinding, allocation concealment, and intention-to-treat analysis.
   • No $\rightarrow$ Observational Design. Proceed to Question 2.

2. How were participants initially categorized or sampled?

   • By Disease/Outcome Status (Cases vs Controls) $\rightarrow$ Case-Control Study. Look for Odds Ratio and watch for recall bias.
   • By Exposure Status (Exposed vs Unexposed) $\rightarrow$ Cohort Study. Look for Relative Risk and watch for loss to follow-up.
   • At a Single Point in Time (No follow-up) $\rightarrow$ Cross-Sectional Study. Measures prevalence.

3. Does early screening increase survival time without reducing total mortality?

   • Yes $\rightarrow$ Lead-Time Bias.

4. Is an extraneous variable linked to exposure AND outcome?

   • Yes $\rightarrow$ Confounding. Correct via matching, stratification, or multivariable regression.