17.1 Statistics & Critical Appraisal
Key Takeaways
- Descriptive statistics summarise data (mean, median, mode, standard deviation, range, interquartile range); the median and IQR are preferred for skewed data, the mean and SD for symmetrical (normal) distributions.
- Inferential statistics generalise from a sample to a population using hypothesis tests (t-test, chi-square, ANOVA) and estimation (confidence intervals); the choice of test depends on data type and distribution.
- Type I (alpha) error is a false positive — rejecting a true null; Type II (beta) error is a false negative — failing to reject a false null; power = 1 − beta, typically targeted at 0.8.
- Diagnostic test performance is described by sensitivity (true positive rate), specificity (true negative rate), positive and negative predictive value, and likelihood ratios; sensitivity and specificity are properties of the test, while predictive values depend on prevalence.
- Correlation (r) measures association; regression models the relationship and predicts outcomes — correlation does not imply causation.
Descriptive Statistics
Descriptive statistics summarise a dataset without inferring beyond it. The choice depends on data type and distribution.
| Measure | Use |
|---|---|
| Mean | Average — for symmetrical (normal) continuous data |
| Median | Middle value — for skewed data (resistant to outliers) |
| Mode | Most common value — for categorical data |
| Standard deviation (SD) | Spread around the mean — for normal data |
| Interquartile range (IQR) | Spread around the median (25th–75th centile) — for skewed data |
| Range | Max to min — simplest spread, sensitive to outliers |
Data Types
- Categorical / qualitative — nominal (no order, e.g. blood group) or ordinal (ordered, e.g. pain score 0–10).
- Numerical / quantitative — discrete (counts) or continuous (measurements).
Skewness
- Normal (symmetrical) distribution — mean ≈ median ≈ mode; use mean and SD.
- Right (positive) skew — long tail to the right; mean > median; use median and IQR (e.g. income, length of stay).
- Left (negative) skew — long tail to the left; mean < median; use median and IQR.
The standard deviation measures how far values typically lie from the mean; in a normal distribution about 95% of values fall within mean ± 2 SD (the empirical rule).
Sampling and the Normal Distribution
The standard error (SE) measures the precision of a sample statistic (e.g. the mean) as an estimate of the population value; SE = SD / √n. As sample size grows, the SE shrinks — estimates become more precise.
The central limit theorem states that, with a sufficiently large sample, the sampling distribution of the mean is approximately normal regardless of the underlying distribution — the basis for many inferential tests.
Inferential Statistics
Inference generalises from a sample to a population. Two complementary approaches:
- Hypothesis testing — compare a null hypothesis (no effect) against an alternative; produce a p-value.
- Estimation — produce a confidence interval for the effect.
Choosing a Test
| Comparison | Data type | Test |
|---|---|---|
| Two independent groups | Continuous, normal | Independent (unpaired) t-test |
| Two related groups (paired) | Continuous, normal | Paired t-test |
| Two groups | Continuous, non-normal | Mann-Whitney U (independent) / Wilcoxon signed-rank (paired) |
| Three or more groups | Continuous, normal | ANOVA |
| Association between two categorical variables | Categorical | Chi-square test |
| Correlation | Continuous | Pearson (normal) / Spearman (non-normal) |
| Predicting an outcome from one or more predictors | Continuous outcome | Linear regression |
| Binary outcome from predictors | Binary outcome | Logistic regression |
The Null Hypothesis and p-values
- Null hypothesis (H₀): no difference / no association.
- Alternative hypothesis (H₁): there is a difference.
- p-value: the probability of the observed result (or more extreme) if H₀ were true.
- Significance level (α): conventionally 0.05. If p < α, reject H₀.
A p-value is not the probability that the null is true, nor the probability the result is due to chance; it is conditional on the null being true.
Type I and Type II Errors
| Error | Definition | Probability |
|---|---|---|
| Type I (alpha) | False positive — rejecting a true null | α (typically 0.05) |
| Type II (beta) | False negative — failing to reject a false null | β |
| Power | 1 − β — the probability of correctly rejecting a false null | Typically targeted at ≥ 0.8 (80%) |
Power is increased by a larger sample size, a larger true effect, and a higher significance level. Underpowered studies may miss a real effect (Type II error). Calculating sample size before a study (power calculation) prevents underpowering.
Confidence Intervals
A 95% confidence interval is the range that, over many samples, would contain the true population value 95% of the time. It conveys both the estimate and its precision:
- A narrow CI indicates a precise estimate (large sample, low variability).
- A wide CI indicates uncertainty.
- A CI that excludes the null value (0 for differences, 1 for ratios) implies statistical significance.
Confidence intervals are increasingly preferred to p-values because they convey effect size and precision, not just significance.
Diagnostic Tests
Evaluating a diagnostic test compares its result with a reference (gold) standard.
| Measure | Definition | Question answered |
|---|---|---|
| Sensitivity | True positive rate: TP / (TP + FN) | How good is the test at identifying disease in those who have it? |
| Specificity | True negative rate: TN / (TN + FP) | How good is the test at ruling disease out in those who don't have it? |
| Positive predictive value (PPV) | TP / (TP + FP) | If the test is positive, how likely is disease? |
| Negative predictive value (NPV) | TN / (TN + FN) | If the test is negative, how likely is the patient disease-free? |
Sensitivity and Specificity vs Predictive Values
- Sensitivity and specificity are properties of the test and do not change with prevalence.
- PPV and NPV depend on the prevalence of disease in the tested population: a positive test is more likely to be a true positive when disease is common; in a low-prevalence population even a specific test yields many false positives.
Likelihood Ratios
The likelihood ratio (LR) combines sensitivity and specificity and is independent of prevalence. LR+ = sensitivity / (1 − specificity); LR− = (1 − sensitivity) / specificity. A high LR+ strongly increases the probability of disease; a low LR− strongly decreases it. LRs are useful because they combine with pre-test probability to give a post-test probability.
SnNout and SpPin
- SnNout — when a test with high Sensitivity is Negative, disease is ruled out.
- SpPin — when a test with high Specificity is Positive, disease is ruled in.
Correlation and Regression
Correlation quantifies the strength and direction of a linear association between two variables, with r from −1 to +1. Pearson's r assumes normality; Spearman's rank is non-parametric. Correlation does not imply causation — a confounder may explain the association.
Regression models the relationship and predicts an outcome. Linear regression predicts a continuous outcome; logistic regression predicts a binary outcome (e.g. disease yes/no). Regression can adjust for confounders and quantify independent effects.
Critical Appraisal — Putting It Together
When appraising a paper:
- Is the question focused (PICO)?
- Is the design appropriate to the question?
- Were the groups comparable at baseline (randomised; matched)?
- Was there blinding and allocation concealment?
- Was the analysis intention-to-treat?
- Is the effect size and its confidence interval clinically meaningful?
- Are the results applicable to your patient?
CASP checklists structure this appraisal (see 14.4).
A study reports the lengths of hospital stay after a new extraction technique, and the data are markedly right-skewed with a few very long stays. Which pair of summary measures is most appropriate?
A research team sets their significance level at 0.05 and their power at 0.80 for a trial of a new bonding agent. What does a power of 0.80 represent?
A new salivary test for oral cancer has high sensitivity. Using the 'SnNout' principle, what does this imply for clinical practice?
A diagnostic test with high specificity is introduced into a low-prevalence screening population. What is the likely effect on the positive predictive value (PPV)?