3.1 Psychometric Principles & Assessment Selection

3.1 Psychometric Principles & Assessment Selection

Core Clinical Principle: Psychometric rigor is the bedrock of ethical, evidence-based speech-language assessment. A clinician must evaluate a standardized instrument's score distributions, reliability, validity, and diagnostic accuracy metrics before drawing diagnostic conclusions or formulating treatment plans.

In Speech-Language Pathology, assessment serves to identify clinical disorders, characterize communicative strengths and deficits, establish baseline performance, and track intervention progress over time. To accomplish these goals, clinicians must possess a sophisticated understanding of measurement theory, classical test theory, and statistical score transformations.

---

1. Classical Test Theory & Standardized Score Distributions

According to Classical Test Theory (CTT), an observed score ($X$) on any clinical instrument is composed of two additive components: the true score ($T$) and measurement error ($E$):

$X = T + E$

Measurement error represents random variability introduced by environmental factors, fatigue, motivation, examiner administration nuances, or psychometric flaws in test construction. Because the true score can never be directly observed, standardized assessments establish normative data by administering tests under uniform, standardized conditions to a representative sample.

The Standard Normal Distribution (Bell Curve)

Norm-referenced standardized tests transform raw scores into standardized scores based on a normal distribution. The normal curve is defined by its mean ($\mu$) and standard deviation ($\sigma$), exhibiting known proportional distributions:

• 68.26% of scores fall within $\pm 1.0$ standard deviation of the mean.
• 95.44% of scores fall within $\pm 2.0$ standard deviations of the mean.
• 99.72% of scores fall within $\pm 3.0$ standard deviations of the mean.

Percentile Ranks vs. Age/Grade Equivalents

• Percentile Ranks indicate the percentage of individuals in the normative sample who scored at or below a given raw score. Percentiles are non-linear; the distance between the 50th and 55th percentile represents a much smaller raw score difference than the distance between the 5th and 10th percentile.
• Age and Grade Equivalents represent the median raw score earned by individuals of a specific age or grade. ASHA strongly discourages the sole use of age/grade equivalents for clinical decision-making. Age equivalents are psychometrically flawed because they assume a linear rate of developmental growth, do not account for score dispersion ($SD$), lead to erroneous conclusions about child performance, and cannot be statistically manipulated.

Standard Error of Measurement (SEM) & Confidence Intervals

No test is perfectly reliable. The Standard Error of Measurement (SEM) quantifies the expected fluctuation of an observed score around the true score due to measurement error. SEM is calculated using the test's standard deviation ($SD$) and its reliability coefficient ($r_{xx}$):

$SEM = SD \times \sqrt{1 - r_{xx}}$

Clinicians use the SEM to construct Confidence Intervals (CIs) around an observed score, reflecting the range of values within which the true score is statistically likely to reside:

• 68% Confidence Interval: $\text{Observed Score} \pm (1.00 \times SEM)$
• 90% Confidence Interval: $\text{Observed Score} \pm (1.645 \times SEM)$
• 95% Confidence Interval: $\text{Observed Score} \pm (1.96 \times SEM)$

Clinical Example: If a client achieves a Standard Score of 73 ($SD=15$) on a language test with a reliability of $r_{xx} = 0.91$, the $SEM = 15 \times \sqrt{1 - 0.91} = 15 \times \sqrt{0.09} = 15 \times 0.3 = 4.5$. The 95% confidence interval is $73 \pm (1.96 \times 4.5) = 73 \pm 8.82$, yielding a confidence band of 64.18 to 81.82. Because this band spans across the clinical deficit threshold (70), the clinician must supplement standardized testing with criterion-referenced and informal measures.

---

2. Psychometric Evaluation: Validity & Reliability

Before selecting an assessment tool, the SLP must examine its psychometric manual to verify established validity and reliability parameters.

Types of Validity (Does the test measure what it purports to measure?)

1. Construct Validity: The degree to which a test measures the theoretical construct or trait it claims to measure (e.g., developmental language competence). Demonstrated through factor analysis and developmental progression of scores.
2. Content Validity: The appropriateness and completeness of test items in sampling the target domain. Requires expert review to ensure all relevant subdomains (phonology, morphology, syntax, semantics, pragmatics) are adequately represented without irrelevant tasks.
3. Criterion-Related Validity: The extent to which test scores correlate with an external criterion or independent measure.
   • Concurrent Validity: Correlation between test scores and another validated measure administered simultaneously.
   • Predictive Validity: The ability of test scores to predict future performance or clinical outcomes on a specified criterion.

Types of Reliability (Is the test consistent and reproducible?)

1. Test-Retest Reliability: Stability of scores over time when the same test is re-administered to the same individuals under identical conditions. Measured via correlation coefficient ($r$).
2. Alternate/Parallel Forms Reliability: Consistency of scores across two different versions of the same test instrument.
3. Inter-Rater & Intra-Rater Reliability:
   • Inter-Rater Reliability: Level of agreement between two or more independent examiners scoring the same performance (measured via point-by-point agreement, Cohen's kappa $\kappa$, or Intraclass Correlation Coefficients ICC).
   • Intra-Rater Reliability: Consistency of a single examiner scoring the same test performance at different points in time.
4. Internal Consistency: The degree to which items within a test or subtest measure the same construct. Evaluated via Split-Half Reliability (Spearman-Brown corrected) or Cronbach's Alpha ($\alpha$).

Psychometric Threshold for Clinical Practice: A clinical diagnostic instrument should possess internal consistency and test-retest reliability coefficients of $r_{xx} \ge 0.80$ for screening and $r_{xx} \ge 0.90$ for individual diagnostic decision-making.

---

3. Diagnostic Accuracy & Epidemiological Metrics

Standardized tests must demonstrate high diagnostic accuracy to correctly discriminate individuals with speech-language disorders from neurotypical individuals.

                     Actual Disorder Present      Actual Disorder Absent
Positive Test Result        True Positive (TP)          False Positive (FP)
Negative Test Result        False Negative (FN)         True Negative (TN)

Sensitivity, Specificity, and Predictive Values

$\text{Sensitivity} = \frac{TP}{TP + FN} \times 100$

• Sensitivity (True Positive Rate): The proportion of individuals with the target disorder who test positive on the instrument. High sensitivity ensures few individuals with disorders are missed (low false negatives).

$\text{Specificity} = \frac{TN}{TN + FP} \times 100$

• Specificity (True Negative Rate): The proportion of neurotypical individuals who test negative on the instrument. High specificity ensures healthy individuals are not misdiagnosed as disordered (low false positives).

Acceptable Clinical Cutoffs: Psychometric standards recommend that both sensitivity and specificity reach \ge 80% (acceptable) and ideally \ge 90% (good to excellent) at recommended score cutoffs.

• Positive Predictive Value (PPV): The probability that a client who tests positive actually has the disorder: $PPV = \frac{TP}{TP + FP}$. PPV is heavily influenced by disorder prevalence in the sample.
• Negative Predictive Value (NPV): The probability that a client who tests negative is truly free of the disorder: $NPV = \frac{TN}{TN + FN}$.
• Likelihood Ratios:
  • Positive Likelihood Ratio ($LR+$): $\frac{\text{Sensitivity}}{1 - \text{Specificity}}$. An $LR+ > 10$ provides strong evidence to confirm the presence of a disorder.
  • Negative Likelihood Ratio ($LR-$): $\frac{1 - \text{Sensitivity}}{\text{Specificity}}$. An $LR- < 0.1$ provides strong evidence to rule out a disorder.

---

4. Assessment Types & Culturally Responsive Selection

Managing Test Bias in Diverse Populations

When evaluating Culturally and Linguistically Diverse (CLD) clients, administering a standardized test outside its normative sample violates test validity:

1. Content Bias: Test items assume vocabulary, concepts, or life experiences specific to mainstream culture.
2. Linguistic/Language Bias: Test scoring penalizes non-standard dialects (e.g., African American English [AAE], Spanish-influenced English [SIE]) or bilingual language structures.
3. Value/Format Bias: Test format assumes familiarity with rapid questioning, displays of knowledge to adults, or computerized testing.

Clinical Action: If a standardized test is administered to a client not represented in the normative sample, standard scores must NOT be reported. The clinician must report performance qualitatively or using criterion-referenced scoring.