9.4 Method Evaluation, Diagnostic Statistics & Point-of-Care Testing
Key Takeaways
Diagnostic sensitivity is TP/(TP+FN) and rules disease OUT when negative; specificity is TN/(TN+FP) and rules disease IN when positive.
Sensitivity and specificity are intrinsic to the test, but positive and negative predictive values depend on disease PREVALENCE, so a good assay used for screening produces mostly false positives.
Before reporting nonwaived results, CLIA requires verification of manufacturer claims for accuracy, precision, reportable range, and reference intervals; a laboratory-modified or laboratory-developed test must additionally ESTABLISH analytical sensitivity and specificity.
The analytical measurement range is the directly measured linear interval; the clinically reportable range extends it by validated dilution.
IQCP is a voluntary CLIA alternative for NONWAIVED testing requiring a risk assessment, a written quality control plan, and ongoing quality assessment; it never applies to waived testing and never removes proficiency testing or competency obligations.
Method Evaluation, Diagnostic Statistics & Point-of-Care Testing
Laboratory Operations includes two topics that are frequently confused: the analytical performance characteristics that describe how well an instrument measures, and the diagnostic statistics that describe how well a result predicts disease. Point-of-care testing is where both meet under a distinct regulatory model.
1. Diagnostic Statistics: The Two-by-Two Table
| Disease Present | Disease Absent | |
|---|---|---|
| Test Positive | True positive (TP) | False positive (FP) |
| Test Negative | False negative (FN) | True negative (TN) |
| Measure | Formula | Plain Meaning |
|---|---|---|
| Diagnostic sensitivity | Of those WITH disease, the proportion the test detects. A highly sensitive test rules disease out when negative. | |
| Diagnostic specificity | Of those WITHOUT disease, the proportion the test correctly calls negative. A highly specific test rules disease in when positive. | |
| Positive predictive value | Of those testing positive, the proportion that truly have disease | |
| Negative predictive value | Of those testing negative, the proportion that truly do not | |
| Efficiency | Overall proportion correctly classified |
The single most tested principle: sensitivity and specificity are intrinsic to the test and do not change with the population. Predictive values depend on prevalence. The same assay used to screen a healthy population produces a much lower positive predictive value than when it is used in a symptomatic population, because most positives in a low-prevalence setting are false positives. This is why a D-dimer with 95% sensitivity is used to EXCLUDE venous thromboembolism in low-probability patients but never to confirm it.
Worked example. A flow cytometry screen for a PNH clone is run on 1,000 patients, 50 of whom truly have PNH. It detects 48 of them and calls 38 of the 950 unaffected patients positive. Sensitivity = 48/50 = 96%. Specificity = 912/950 = 96%. Positive predictive value = 48/(48+38) = 56%. Despite excellent sensitivity and specificity, fewer than 6 in 10 positives are real, because the disease is uncommon.
2. Analytical Performance Characteristics
| Characteristic | Definition | How It Is Established |
|---|---|---|
| Analytical sensitivity (limit of detection) | The lowest concentration reliably distinguished from zero | Replicate analysis of blank and low-level material |
| Analytical specificity | Freedom from interference by substances other than the analyte | Interference studies with hemolysate, bilirubin, lipid, drugs |
| Precision | Reproducibility, expressed as within-run and between-run CV | Repeated measurement of controls over at least 20 days |
| Accuracy / bias | Agreement with a reference method or reference material | Method comparison against a comparative method, regression analysis |
| Linearity and analytical measurement range | The interval over which results are directly proportional to concentration | Dilution series spanning the claimed range |
| Clinically reportable range | The AMR extended by permitted dilution or concentration | AMR multiplied by the validated dilution factor |
| Carryover | Contamination of one specimen by the preceding one | High-low-low sequence testing |
| Reference interval verification | Confirming that a published interval applies to this population | Analyzing a modest number of healthy local subjects (commonly 20) and confirming acceptable agreement |
For nonwaived tests, CLIA requires the laboratory to verify manufacturer performance claims for accuracy, precision, reportable range, and reference intervals before reporting patient results. If the laboratory modifies an FDA-cleared method or develops its own, it must additionally establish analytical sensitivity, analytical specificity, and other performance characteristics.
3. Point-of-Care Testing
Point-of-care testing is performed near the patient by staff who are usually not laboratory-trained, which shifts the quality burden from the instrument to the management program.
Hematology and Coagulation Point-of-Care Devices
| Device Category | Typical Measurement Principle | Common Pitfalls |
|---|---|---|
| Portable hemoglobinometer | Reflectance or transmission photometry on a microcuvette | Poor capillary technique, milking the finger, interstitial fluid dilution; first drop must be wiped away |
| Point-of-care INR meter | Capillary whole blood clotting on a test strip with an electrochemical or optical endpoint | Not validated for lupus anticoagulant patients or for direct oral anticoagulants; INR agreement degrades above about 4.0 |
| Activated clotting time (ACT) | Whole blood clotting after contact activation, used for high-dose heparin during bypass and catheterization | Insensitive at low heparin doses; hemodilution and hypothermia prolong it |
| Point-of-care platelet function analyzers | Shear-induced platelet adhesion or aggregation on a cartridge | Dependent on platelet count and hematocrit; low values in thrombocytopenia or anemia are not necessarily drug effects |
| Handheld blood gas and electrolyte cartridges | Micro-electrode arrays | Cartridge storage temperature and air bubbles |
Regulatory Model
- CLIA complexity. Waived tests carry the lowest regulatory burden but still require adherence to manufacturer instructions and a CLIA certificate of waiver. Moderate and high complexity point-of-care tests carry the same personnel, quality control, proficiency testing, and competency obligations as the main laboratory.
- Individualized Quality Control Plan (IQCP). An optional CLIA-recognized alternative for nonwaived testing that permits a reduced control frequency only after a documented risk assessment covering specimen, environment, reagent, testing personnel, and test system, plus a written quality control plan and ongoing quality assessment. IQCP does not apply to waived testing.
- Program requirements include a designated point-of-care coordinator, documented operator training and competency, lot-to-lot verification, connectivity so results reach the permanent record, and correlation studies between point-of-care devices and the central laboratory whenever both report the same analyte.
A screening assay has a diagnostic sensitivity of 96% and a specificity of 96%. It is applied first in a specialty clinic where disease prevalence is 40%, and then in a general screening population where prevalence is 1%. What happens to the performance measures?
Sensitivity and specificity stay the same, but the POSITIVE PREDICTIVE VALUE falls sharply in the low-prevalence population because most positives there are false positives
Sensitivity and specificity both fall in the screening population
Positive predictive value is unchanged, because it is an intrinsic property of the assay
Negative predictive value falls in the low-prevalence population
A new hematology analyzer is being brought into service and the laboratory needs to define the interval over which reported results are directly proportional to concentration. Which performance characteristic is being established, and how?
Analytical sensitivity, by replicate testing of blank material
Linearity and the analytical measurement range, by analyzing a dilution series that spans the manufacturer's claimed range and confirming proportionality
Carryover, by running a high specimen followed by two low specimens
Reference interval verification, by testing 20 healthy local subjects
Which statement about the Individualized Quality Control Plan (IQCP) is correct?
It is mandatory for all waived point-of-care testing
It eliminates the requirement for proficiency testing on the affected analytes
It is an optional CLIA-recognized alternative for NONWAIVED testing that permits a reduced control frequency only after a documented risk assessment, a written quality control plan, and ongoing quality assessment
It replaces operator competency assessment for point-of-care personnel
A point-of-care INR meter reports 2.4 on a patient whose plasma INR from the central laboratory is 4.6. The patient has a known lupus anticoagulant. What is the best explanation?
The central laboratory result is falsely elevated because lupus anticoagulants do not affect the prothrombin time
Point-of-care meters are more accurate at high INR values and should be used preferentially
The discrepancy proves a specimen mix-up and both results should be discarded
Point-of-care INR meters use thromboplastin-coated strips that respond differently to a lupus anticoagulant, and their agreement with plasma methods degrades at high INR values, so the plasma INR should be used for dosing
Sections you finish are checked off in the contents.