31.3 The Physician as Researcher II: Biostatistics (Data Types, Distributions, Central Tendency and Dispersion, Confidence Intervals, Hypothesis Testing and Test Selection) and Screening Tests (Validity, Predictive Values, Likelihood Ratios, Biases and Program Criteria)

Why this area matters

The TOS asks you to compute rates and ratios and apply biostatistics to research findings. Screening programs such as newborn screening, cervical screening, TB case-finding and NCD risk screening depend on understanding test performance. Many board items are short calculations from a 2x2 table, or a choice of the correct statistical test.

Data Types and Probability Distributions

Selecting appropriate statistical methods requires precise identification of variable types and underlying distribution shapes.

Scales of Measurement

1. Qualitative (Categorical):
   • Nominal: Unordered categories without intrinsic mathematical hierarchy (e.g., biological sex, ABO blood type, mortality status [alive/dead]).
   • Ordinal: Ordered categorical ranks with non-standardized or unequal intervals between steps (e.g., cancer staging I-IV, Glasgow Coma Scale, NYHA heart failure classes, Likert pain scales).
2. Quantitative (Numerical):
   • Discrete: Integer counts that cannot take fractional values (e.g., number of clinic visits, lifetime pregnancies, discrete seizure episodes).
   • Continuous: Real numbers along an unbroken continuum with meaningful decimal fractions (e.g., serum creatinine, blood pressure, core body temperature).

The Gaussian (Normal) Distribution

The normal distribution is a symmetrical, unimodal, bell-shaped distribution where the three measures of central tendency coincide exactly:

$\text{Mean} = \text{Median} = \text{Mode}$

The Empirical Rule (68-95-99.7 Rule)

In a perfectly Gaussian distribution governed by mean ($\mu$) and standard deviation ($\sigma$):

• $\mu \pm 1\sigma$ contains $68.27\%$ of all observations.
• $\mu \pm 1.96\sigma$ contains exactly $95.00\%$ of observations (often approximated as $\mu \pm 2\sigma$ containing $95.45\%$).
• $\mu \pm 3\sigma$ contains $99.73\%$ of all observations.

Skewed Distributions and Measures of Central Tendency

When data violate normality, extreme outliers pull distribution parameters unevenly:

• Positively Skewed (Right-Skewed):
  • The distribution has a prolonged tail extending to the positive (right) side.
  • Outliers pull the mean upward: $\text{Mode} < \text{Median} < \text{Mean}$.
  • Preferred summary metrics: Median and Interquartile Range (IQR = Q3 - Q1), which are resistant to extreme outliers.
  • Clinical examples: Length of hospital stay, ICU ventilation hours, serum viral loads, annual healthcare expenditures.
• Negatively Skewed (Left-Skewed):
  • The distribution has a prolonged tail extending to the negative (left) side.
  • Outliers pull the mean downward: $\text{Mean} < \text{Median} < \text{Mode}$.
  • Clinical example: Age at death from natural causes in developed nations.

Measures of dispersion, standard error and confidence intervals

• Range: maximum minus minimum.
• Interquartile range: the 25th to 75th percentile, used with the median for skewed data.
• Variance and standard deviation (SD): spread around the mean, used for normally distributed data.
• Coefficient of variation: SD divided by the mean, used to compare variability across different scales.
• Standard error of the mean: SD divided by the square root of n. It measures the precision of the sample mean and shrinks as the sample grows.
• The 95% confidence interval is about the estimate plus or minus 1.96 standard errors. It is the range that would contain the true population value in 95% of repeated samples.
  • Narrow intervals mean precise estimates.
  • If the 95% CI of a difference includes 0, or the CI of a ratio (RR, OR) includes 1, the result is not statistically significant at the 0.05 level.
• Statistical significance is not clinical importance. A tiny effect can be significant in a huge sample, so look at the effect size and the CI.

Hypothesis Testing, Decision Errors & Statistical Power

Hypothesis testing evaluates whether empirical data provide sufficient evidence to reject a baseline assertion.

• Null Hypothesis ($H_0$): States that there is no true difference between study groups or no association between exposure and outcome ($H_0: \mu_1 = \mu_2$).
• Alternative Hypothesis ($H_a$ or $H_1$): States that a true difference or association exists ($H_a: \mu_1 \ne \mu_2$).

The Error Matrix in Hypothesis Testing

• Type I Error ($\alpha$): Rejecting the null hypothesis when it is genuinely true (finding a difference that does not exist). The significance threshold is conventionally set at $\alpha = 0.05$ (5%). The $p$-value represents the exact probability of obtaining a test statistic as extreme as or more extreme than observed, assuming $H_0$ is true. If $p < \alpha$, $H_0$ is rejected.
• Type II Error ($\beta$): Failing to reject the null hypothesis when it is genuinely false (failing to detect a real difference). Conventionally set at $\beta = 0.20$ (20%).
• Statistical Power ($1 - \beta$): The probability of correctly rejecting a false null hypothesis (detecting a true treatment difference). The acceptable standard is $\ge 80\%$ ($0.80$).

Determinants of Statistical Power

Statistical power is mathematically increased by:

1. Increasing sample size ($N$): Enlarges denominator degrees of freedom and narrows standard error.
2. Increasing effect size: A large difference between treatment arms is easier to detect than a subtle one.
3. Decreasing measurement variance / standard deviation ($\sigma$): Minimizes background biological noise.
4. Increasing the alpha threshold ($\alpha$): Moving $\alpha$ from 0.01 to 0.05 increases power, but raises the risk of Type I errors.

---

Inferential Statistical Test Selection Grid

Choosing the correct bivariate statistical test depends on the scale of measurement, data distribution, and group independence:

---

Correlation and Regression

• Pearson Correlation Coefficient ($r$): Quantifies the linear association between two continuous, normally distributed variables. Ranges from $-1.0$ (perfect inverse linear correlation) to $+1.0$ (perfect direct linear correlation), with $0.0$ indicating no linear correlation.
• Spearman Rank Correlation ($\rho$): Non-parametric equivalent of Pearson correlation, evaluating monotonic relationships between non-normally distributed or ordinal variables.
• Coefficient of Determination ($R^2$): Calculated as $r^2$; represents the proportion of total variance in the dependent variable explained by the independent variable.
• Linear vs. Logistic Regression: Linear regression models continuous outcome variables ($y = \beta_0 + \beta_1 x$). In contrast, Logistic Regression models dichotomous binary outcomes (e.g., survival vs. death; cured vs. relapsed) and generates Adjusted Odds Ratios controlling for multiple clinical confounders.

Screening Test Validity, Predictive Values & Likelihood Ratios

Screening identifies unrecognized disease in asymptomatic individuals to allow early therapeutic intervention.

Validity Metrics: Sensitivity and Specificity

• Sensitivity (True Positive Rate): The probability that a diseased individual tests positive: $\frac{TP}{TP + FN}$.
  • Mnemonic: SnNOut: A test with high Sensitivity, when Negative, rules Out the disease.
  • Clinical application: Initial public health screening where missing a case carries catastrophic consequences (e.g., ELISA screening for HIV; rapid molecular testing for presumptive tuberculosis; neonatal metabolic screening under RA 9288).
• Specificity (True Negative Rate): The probability that a non-diseased individual tests negative: $\frac{TN}{TN + FP}$.
  • Mnemonic: SpPIn: A test with high Specificity, when Positive, rules In the disease.
  • Clinical application: Confirmatory diagnostic verification before initiating toxic, invasive, or stigmatizing therapies (e.g., the confirmatory rapid HIV diagnostic algorithm, which replaced the Western blot in the Philippines; histopathological biopsy for malignancy).

Predictive Values and the Impact of Population Prevalence

• Positive Predictive Value (PPV): The probability that an individual with a positive test result genuinely possesses the disease: $\frac{a}{a + b}$.
• Negative Predictive Value (NPV): The probability that an individual with a negative test result is genuinely free of the disease: $\frac{d}{c + d}$.

Cardinal Epidemiologic Principle: Prevalence Dependence

• Sensitivity and Specificity are intrinsic characteristics of the test methodology itself; they do not change when the test is applied to populations with differing disease prevalence.
• PPV and NPV are fundamentally dependent on disease prevalence:
  • As population prevalence increases: PPV increases markedly, while NPV decreases slightly.
  • As population prevalence decreases: PPV plummets (meaning the vast majority of positive tests represent false positives), while NPV approaches 100%.

Likelihood Ratios (LR)

Likelihood ratios quantify the clinical diagnostic power of a test result independently of disease prevalence:

• Positive Likelihood Ratio (LR+): The ratio of the probability of a positive test in a diseased individual compared to a non-diseased individual:

$LR+ = \frac{\text{Sensitivity}}{1 - \text{Specificity}} = \frac{\text{True Positive Rate}}{\text{False Positive Rate}}$

• Negative Likelihood Ratio (LR-): The ratio of the probability of a negative test in a diseased individual compared to a non-diseased individual:

$LR- = \frac{1 - \text{Sensitivity}}{\text{Specificity}} = \frac{\text{False Negative Rate}}{\text{True Negative Rate}}$

• Clinical Utility Thresholds:
  • $LR+ > 10$: Substantially increases disease probability; excellent for ruling in disease.
  • $LR- < 0.1$: Substantially decreases disease probability; excellent for ruling out disease.
  • $LR = 1.0$: Completely uninformative; post-test probability equals pre-test probability.
  • Mathematical translation: $\text{Post-test Odds} = \text{Pre-test Odds} \times LR$.

---

Screening Biases: Methodological Pitfalls

Apparent survival improvements attributed to screening initiatives are frequently artifacts of methodological bias:

1. Lead-Time Bias: Occurs when screening identifies disease earlier in its subclinical phase without altering the natural history or final time of death. The chronological duration between diagnosis and death appears prolonged, creating the false illusion of therapeutic efficacy.
2. Length-Time Bias (Prognostic Selection Bias): Screening tests preferentially detect indolent, slowly progressive disease variants because their asymptomatic preclinical phase is prolonged. Aggressive, rapidly fatal malignancies have brief preclinical phases and emerge between screening rounds (interval cancers). Consequently, screened cohorts demonstrate spuriously superior survival rates.
3. Overdiagnosis Bias: The detection and treatment of indolent pseudodisease—abnormalities meeting microscopic or radiographic criteria for disease that would never have produced symptoms or shortened life expectancy during the patient's natural lifespan (e.g., low-grade prostate carcinoma, thyroid microcarcinomas).
4. Volunteer Bias (Self-Selection Bias): Individuals who voluntarily participate in screening programs typically possess higher health literacy, healthier lifestyle behaviors, and greater therapeutic adherence than non-participants, confounding outcome comparisons.

More on screening tests and programs

• ROC curve: a plot of sensitivity against 1 minus specificity across cut-offs. The area under the curve measures overall accuracy (1.0 is perfect and 0.5 is useless).
  • Lowering the cut-off for positivity raises sensitivity and lowers specificity, as with a lower fasting glucose threshold.
• Combining tests:
  • Serial (sequential) testing: positive only if both tests are positive. It increases specificity and is used to confirm diagnoses, as in the HIV testing algorithm.
  • Parallel testing: positive if either test is positive. It increases sensitivity and is used when missing a case is dangerous.
• Reliability (precision) is the consistency of repeated results, between observers (kappa statistic) or within one observer. Validity is closeness to the truth. A test can be reliable but not valid.
• Wilson-Jungner criteria for a screening program:
  • the condition is an important health problem;
  • it has a recognizable latent or early stage;
  • an acceptable, accurate test exists;
  • effective treatment for early disease is available;
  • facilities for diagnosis and treatment exist;
  • the policy on whom to treat is agreed;
  • the cost is balanced against overall spending;
  • case-finding is continuous.
• Examples in the Philippines: newborn screening, VIA or HPV cervical screening, BP and glucose screening under PhilPEN, and TB screening of contacts and high-risk groups.