1.3 Measures of Association & Diagnostic Accuracy
Key Takeaways
- Relative risk compares disease incidence in exposed vs. unexposed groups in cohort studies, while odds ratio compares exposure odds in case-control studies.
- The rare disease assumption states that when disease prevalence is low (typically under 5%), the odds ratio closely approximates the relative risk.
- Sensitivity (SNOUT) rules out disease when negative, whereas specificity (SPIN) rules in disease when positive; both are intrinsic test properties unaffected by prevalence.
- Positive predictive value increases and negative predictive value decreases as the disease prevalence in the tested population increases.
- Likelihood ratios combine sensitivity and specificity to modify pre-test odds to post-test odds, making them clinically superior and independent of prevalence.
Measures of Association & Diagnostic Accuracy
In clinical practice, physicians must continuously evaluate the strength of associations between risk factors and diseases, as well as the performance of diagnostic tests. Mastering the calculations and clinical applications of measures of association, predictive values, likelihood ratios, and receiver operating characteristic curves is essential for clinical decision-making and is a core focus of the USMLE Step 3.
Measures of Association
Measures of association quantify the relationship between an exposure and an outcome. The choice of metric depends on the study design.
Relative Risk (RR)
Relative Risk (also known as the risk ratio) is used in prospective study designs, such as cohort studies and randomized controlled trials, where the incidence of disease can be directly measured.
- Interpretation:
- $\text{RR} = 1$: No association.
- $\text{RR} > 1$: Increased risk of outcome in the exposed group.
- $\text{RR} < 1$: Decreased risk (protective effect) in the exposed group.
- Example: Consider a cohort where 10% of exposed vs. 2% of unexposed develop an outcome; the Relative Risk is 10%/2% = 5.0.
Odds Ratio (OR)
The Odds Ratio is typically used in case-control studies, where the incidence of disease cannot be directly calculated because the proportion of cases and controls is determined by the investigator. It compares the odds of exposure among the cases to the odds of exposure among the controls.
- The Rare Disease Assumption: When a disease is rare in the population (typically prevalence $< 5%$), the odds ratio closely approximates the relative risk. This allows researchers to use case-control studies to draw conclusions about relative risk.
- Example: For an OR, if exposure odds are 4 in cases and 0.125 in controls, the Odds Ratio is 4/0.125 = 32.0.
Diagnostic Accuracy and the 2x2 Table
Diagnostic tests are evaluated using a standard 2x2 table comparing test results against a gold standard reference diagnosis.
| Test Result \ Disease Status | Disease Present | Disease Absent |
|---|---|---|
| Test Positive | True Positive (TP) | False Positive (FP) |
| Test Negative | False Negative (FN) | True Negative (TN) |
Sensitivity
Sensitivity is the probability that the test is positive in a patient who actually has the disease.
- Clinical Utility: High-sensitivity tests are used for screening. A negative result in a highly sensitive test helps rule out the disease (remember the mnemonic SNOUT: Sensitivity rules OUT).
Specificity
Specificity is the probability that the test is negative in a patient who does not have the disease.
- Clinical Utility: High-specificity tests are used for confirmation. A positive result in a highly specific test helps rule in the disease (remember the mnemonic SPIN: SPecificity rules IN).
Predictive Values and the Influence of Prevalence
Unlike sensitivity and specificity, which are intrinsic, predictive values depend heavily on disease prevalence in the tested population.
- Positive Predictive Value (PPV): The probability that a patient actually has the disease given a positive test result.
- Negative Predictive Value (NPV): The probability that a patient does not have the disease given a negative test result.
- Prevalence Effects:
- As disease prevalence increases, the PPV increases (because there are more true positives and fewer false positives in the population), while the NPV decreases.
- As disease prevalence decreases, the PPV decreases (even a highly specific test will yield many false positives when the disease is rare), while the NPV increases.
Likelihood Ratios (LRs)
Likelihood Ratios combine sensitivity and specificity into a single measure that is independent of disease prevalence. They indicate how much a test result shifts the probability of having the disease from the pre-test probability to the post-test probability.
- Positive Likelihood Ratio ($\text{LR}^+$): Indicates how much the odds of disease increase when the test is positive. An $\text{LR}^+ > 10$ represents a strong diagnostic test that significantly increases the probability of disease.
- Negative Likelihood Ratio ($\text{LR}^-$): Indicates how much the odds of disease decrease when the test is negative. An $\text{LR}^- < 0.1$ represents a strong test that significantly decreases the probability of disease.
- Clinical Application: Likelihood ratios are clinically superior to predictive values because they can be applied directly to a patient's individual pre-test probability to calculate post-test probability using the formula: $\text{Pre-test Odds} \times \text{Likelihood Ratio} = \text{Post-test Odds}$. For example, if a patient has a 20% pre-test probability of coronary artery disease (pre-test odds = 0.25) and undergoes a stress test with an $\text{LR}^+$ of 8.0, their post-test odds become $0.25 \times 8.0 = 2.0$, which corresponds to a post-test probability of $2.0 / (2.0 + 1) = 66.7%$.
Receiver Operating Characteristic (ROC) Curves
Many clinical tests yield continuous numerical results (e.g., troponin) requiring a cutoff threshold.
Shifting Cutoffs
Adjusting the cutoff threshold creates a trade-off between sensitivity and specificity:
- Lowering the Cutoff: Increases sensitivity (more true positives are captured) but decreases specificity (more false positives are introduced).
- Raising the Cutoff: Increases specificity (fewer false positives are generated) but decreases sensitivity (more true positives are missed).
The ROC Plot
An ROC curve plots Sensitivity (y-axis) against 1 - Specificity (x-axis).
- Area Under the Curve (AUC): Measures the overall accuracy of the test.
- $\text{AUC} = 1.0$: A perfect test with no overlap between diseased and healthy populations.
- $\text{AUC} = 0.5$: A test with no diagnostic value, equivalent to a random guess.
- The closer the curve clings to the top-left corner, the higher the diagnostic accuracy of the test.
A new rapid antigen screening test for influenza is evaluated in two different settings. Setting A is an emergency department during peak winter influenza season, where the prevalence of influenza is 30%. Setting B is an outpatient primary care clinic during the summer, where the prevalence of influenza is 1%. The test has an intrinsic sensitivity of 90% and specificity of 95% in both settings. Which of the following statements correctly compares the performance of the test in these two settings?
A laboratory develops an assay to measure serum levels of a novel biomarker to screen for early-stage pancreatic cancer. The biomarker is measured on a continuous scale. If the laboratory decides to raise the diagnostic cutoff value for a "positive" test result to make it more stringent, how will the sensitivity and specificity of the assay be affected?
A diagnostic test for coronary artery disease has a sensitivity of 80% and a specificity of 90%. What is the positive likelihood ratio (LR+) of this test?