17.2 Medical Statistics, Diagnostic Tests & Critical Appraisal
Key Takeaways
Sensitivity is the proportion of people with the disease who test positive (TP/(TP+FN)); specificity is the proportion without the disease who test negative (TN/(TN+FP)).
Positive and negative predictive values depend on disease prevalence, whereas sensitivity and specificity are properties of the test itself.
The number needed to treat is 1 divided by the absolute risk reduction (the control event rate minus the experimental event rate).
Case-control studies compare exposures in cases and controls and report odds ratios, while cohort studies follow exposed and unexposed groups and report relative risks.
A 95% confidence interval for a relative risk or odds ratio that includes 1 (or for a difference that includes 0) indicates a result that is not statistically significant at the 5% level.
Medical Statistics, Diagnostic Tests & Critical Appraisal
Diagnostic Test Accuracy
Use a 2 x 2 table comparing the test with a reference standard:
| Disease Present | Disease Absent | |
|---|---|---|
| Test Positive | True positive (TP) | False positive (FP) |
| Test Negative | False negative (FN) | True negative (TN) |
| Measure | Formula | Meaning |
|---|---|---|
| Sensitivity | TP / (TP + FN) | Proportion of people with the disease who test positive |
| Specificity | TN / (TN + FP) | Proportion of people without the disease who test negative |
| Positive predictive value (PPV) | TP / (TP + FP) | Probability that a positive result is a true positive |
| Negative predictive value (NPV) | TN / (TN + FN) | Probability that a negative result is a true negative |
| Positive likelihood ratio | Sensitivity / (1 − specificity) | How much a positive result increases the odds of disease |
| Negative likelihood ratio | (1 − sensitivity) / specificity | How much a negative result decreases the odds of disease |
- Sensitivity and specificity are properties of the test; predictive values change with prevalence. In a low-prevalence population (screening), even a specific test produces many false positives, so the PPV falls.
- SnNout: a highly sensitive test, when negative, rules a condition out (for example D-dimer for low-probability VTE). SpPin: a highly specific test, when positive, rules a condition in.
Measures of Risk & Treatment Effect
Using the control event rate (CER) and experimental event rate (EER):
| Measure | Formula |
|---|---|
| Absolute risk reduction (ARR) | CER − EER |
| Relative risk (RR) | EER / CER |
| Relative risk reduction (RRR) | (CER − EER) / CER, or 1 − RR |
| Number needed to treat (NNT) | 1 / ARR (round up to a whole number) |
| Number needed to harm (NNH) | 1 / absolute risk increase of an adverse effect |
Worked example: a stroke occurs in 10% of the control group and 6% of the treated group. ARR = 0.10 − 0.06 = 0.04; RR = 0.06 / 0.10 = 0.6; RRR = 40%; NNT = 1 / 0.04 = 25. Relative risk reductions sound impressive; absolute figures and NNT help patients understand benefit.
- Odds ratio (OR): odds of exposure in cases divided by odds in controls; used in case-control studies and logistic regression; approximates the relative risk when the outcome is rare.
- Hazard ratio: the relative rate of an event over time in survival analysis.
- Incidence is the number of new cases in a population over a period; prevalence is the proportion with the condition at a point in time (prevalence is roughly incidence multiplied by average duration).
Study Designs & the Hierarchy of Evidence
| Design | Description | Strengths & Weaknesses |
|---|---|---|
| Systematic review and meta-analysis | Combines all relevant studies; results shown in a forest plot | Highest level if good-quality trials; heterogeneity and publication bias (funnel plot asymmetry) are problems |
| Randomised controlled trial (RCT) | Random allocation to intervention or control | Best design for treatment effects; randomisation balances known and unknown confounders; allocation concealment, blinding and intention-to-treat analysis reduce bias |
| Cohort study | Follows exposed and unexposed groups forward in time | Measures incidence and relative risk; good for rare exposures and multiple outcomes; slow and costly; loss to follow-up |
| Case-control study | Compares past exposures in people with a disease (cases) and without (controls) | Quick, good for rare diseases; gives odds ratios; prone to recall and selection bias |
| Cross-sectional survey | Measures exposure and outcome at one time | Measures prevalence; cannot show cause and effect |
| Case series / case report | Descriptions of patients | Generate hypotheses only |
| Ecological study | Compares populations rather than individuals | Risk of the ecological fallacy |
Phases of drug trials: phase I (safety, dose, often healthy volunteers), phase II (efficacy and side effects in patients), phase III (large comparative RCTs before licensing), phase IV (post-marketing surveillance, including Yellow Card reporting).
Bias, Confounding & Screening Pitfalls
- Selection bias: the study population does not represent the target population.
- Information (measurement) bias: recall bias, observer bias, misclassification.
- Confounding: a third factor linked to both exposure and outcome (for example smoking confounding an association between coffee and lung cancer). Control it by randomisation, restriction, matching, stratification or multivariable analysis.
- Screening biases: lead-time bias (earlier diagnosis seems to lengthen survival without changing the time of death), length-time bias (screening preferentially detects slow-growing disease) and overdiagnosis.
Statistical Significance & Errors
- p-value: the probability of results at least as extreme as those observed if the null hypothesis were true; p below 0.05 is conventionally "significant". It does not measure the size or importance of an effect.
- 95% confidence interval: the range likely to contain the true value. For a ratio (RR, OR, HR), an interval including 1 is not significant; for a difference, an interval including 0 is not significant. Narrow intervals mean more precise estimates.
- Type I error (alpha): a false positive, rejecting a true null hypothesis (usually set at 5%).
- Type II error (beta): a false negative, failing to detect a real effect; power = 1 − beta (usually 80–90%) and increases with sample size and effect size.
Choosing a Statistical Test
| Data | Two Independent Groups | Paired Data | Association |
|---|---|---|---|
| Continuous, normally distributed | Unpaired (Student's) t-test; ANOVA for 3 or more groups | Paired t-test | Pearson's correlation |
| Continuous non-normal or ordinal | Mann-Whitney U test | Wilcoxon signed-rank test | Spearman's rank correlation |
| Categorical (proportions) | Chi-squared test (Fisher's exact test for small numbers) | McNemar's test | Chi-squared |
Distributions
- Normal distribution: symmetrical; mean = median = mode; about 68% of values lie within 1 standard deviation, 95% within about 2 SD and 99.7% within 3 SD.
- Positive (right) skew: a long tail to the right; mean > median > mode. Negative skew: mean < median < mode. Use the median and interquartile range for skewed data.
- Standard deviation describes the spread of the data; the standard error of the mean (SD divided by the square root of n) describes the precision of the mean estimate.
A new blood test for pancreatic cancer is evaluated in 200 people: 100 with cancer and 100 without. It is positive in 90 of those with cancer and in 20 of those without. What are the sensitivity and specificity?
Sensitivity 82%, specificity 90%
Sensitivity 90%, specificity 82%
Sensitivity 80%, specificity 90%
Sensitivity 90%, specificity 80%
In a trial, 20% of patients given placebo and 15% of patients given a new drug have a heart attack over 5 years. How many patients need to be treated for 5 years to prevent one heart attack?
5
20
25
4
Researchers want to know whether exposure to a particular solvent increases the risk of a very rare liver cancer. They identify 60 patients with the cancer and 240 matched people without it, and compare their past occupational exposure. Which measure of association should they report?
Relative risk
Number needed to treat
Odds ratio
Incidence rate
Sections you finish are checked off in the contents.