9.3 Paired t-Tests & Non-Parametric Alternatives

9.3 Paired t-Tests & Non-Parametric Alternatives

Quick Summary: While independent two-sample tests compare separate groups, many continuous improvement studies involve dependent, repeated, or matched observations—such as evaluating the exact same machines Before and After a maintenance kaizen, or having two appraisers inspect identical components. The Paired $t$-test isolates within-unit changes by computing difference scores ($d_i = x_{1i} - x_{2i}$), subtracting out background part-to-part variation and dramatically boosting statistical power. When continuous data severely violates the Gaussian normality assumption and cannot be transformed, practitioners deploy non-parametric tests (Mann-Whitney U, Wilcoxon Signed-Rank, Kruskal-Wallis) to test medians and ranked data rather than means.

---

The Engineering Power of Dependent (Paired) Sampling

In standard manufacturing, healthcare, and administrative processes, natural unit-to-unit variation is often massive:

• In a machining study, casting porosity and alloy density vary naturally from block to block.
• In a clinical study, metabolic rates and blood chemistry vary naturally from patient to patient.
• In loan underwriting, applicant financial history varies widely across applications.

If an engineering team tests a process improvement using an Independent Two-Sample $t$-test—measuring 20 parts made with Setup A and 20 completely different parts made with Setup B—the huge natural casting-to-casting variation enters the error term ($s_p$). The signal of the improvement is drowned out by the background noise of the parts, causing an underpowered test and a Type II Error (missed detection).

How Pairing Eliminates Background Noise

By taking paired measurements on the exact same physical units under both conditions (or using identically matched twins), the background unit-to-unit variation is mathematically subtracted out:

                    How Pairing Eliminates Background Noise

   INDEPENDENT SAMPLES                           PAIRED SAMPLES
  ┌─────────────────────────────┐               ┌─────────────────────────────┐
  │ Shift A parts vs.           │               │ Same parts measured BEFORE  │
  │ Shift B parts               │               │ and AFTER maintenance       │
  ├─────────────────────────────┤               ├─────────────────────────────┤
  │ High part-to-part variation │               │ Part variation cancels out: │
  │ inflates standard error!    │               │ di = x_After - x_Before     │
  │ Lower Statistical Power     │               │ High Statistical Power      │
  └─────────────────────────────┘               └─────────────────────────────┘

Common Six Sigma paired applications include:

1. Before vs. After Studies: Measuring cycle times, temperature, or emissions on the identical set of machines before and after a Lean kaizen event.
2. Measurement System Analysis (MSA): Appraiser 1 and Appraiser 2 measuring the identical 20 production samples to evaluate appraiser bias.
3. Split-Lot / Matched Testing: Applying Coating Formulation A to one half of a silicon wafer and Coating Formulation B to the other half of the identical wafer.

---

Mathematical Formulation of the Paired t-Test

The Paired $t$-test is mathematically identical to a One-Sample $t$-test performed on the single column of difference scores ($d_i$):

$d_i = x_{1i} - x_{2i}$

Hypotheses Formulation

• Null Hypothesis ($H_0$): $\mu_d = \Delta_0$ (Typically $H_0: \mu_d = 0$, asserting zero mean difference between paired conditions).
• Alternative Hypothesis ($H_a$): $\mu_d \ne 0$ (Two-tailed), $\mu_d > 0$, or $\mu_d < 0$ (One-tailed).

Test Statistic & Formulas

$t_{\text{calc}} = \frac{\bar{d} - \Delta_0}{s_d / \sqrt{n}}$

Where:

• $\bar{d} = \frac{\sum_{i=1}^n d_i}{n} = $ Arithmetic mean of the differences
• $s_d = \sqrt{\frac{\sum_{i=1}^n (d_i - \bar{d})^2}{n - 1}} = $ Sample standard deviation of the differences
• $n = $ Number of pairs (NOT the total number of raw observations $2n$)
• $SE_d = \frac{s_d}{\sqrt{n}} = $ Standard error of the mean difference
• $\mathbf{Degrees\ of\ Freedom:}\ df = n - 1$

Exam Watchout: If 15 parts are measured Before and After (giving 30 total measurements), the degrees of freedom is $df = 15 - 1 = 14$. A common distractor on the CSSC exam is $df = 30 - 2 = 28$, which erroneously applies the independent two-sample formula.

---

Step-by-Step Worked Example: Paired t-Test

Scenario: A Six Sigma Green Belt at an electronics manufacturing plant designs a new ergonomic hand fixture to reduce wiring harness assembly cycle time. The team times $n = 10$ assembly technicians using the baseline standard fixture ($T_{\text{Base}}$) and the new ergonomic fixture ($T_{\text{New}}$). Assembly times are recorded in seconds:

Test whether the new ergonomic fixture achieves a statistically significant reduction in assembly cycle time at $\alpha = 0.05$.

Step 1: State the Hypotheses

• $H_0: \mu_d \ge 0$ (New fixture does not reduce assembly cycle time)
• $H_a: \mu_d < 0$ (New fixture significantly reduces assembly cycle time; one-tailed lower test)

Step 2: Compute Mean and Standard Deviation of Differences

• $\bar{d} = \frac{\sum d_i}{n} = \frac{-42.0}{10} = -4.20\text{ seconds}$
• $s_d = \sqrt{\frac{\sum (d_i - \bar{d})^2}{n - 1}} = \sqrt{\frac{9.260}{10 - 1}} = \sqrt{\frac{9.260}{9}} = \sqrt{1.0289} = 1.0143\text{ seconds}$

Step 3: Calculate Standard Error and Test Statistic ($t_{\text{calc}}$)

• Standard Error: $SE_d = \frac{s_d}{\sqrt{n}} = \frac{1.0143}{\sqrt{10}} = \frac{1.0143}{3.1623} = 0.3208\text{ seconds}$
• $t_{\text{calc}} = \frac{\bar{d} - 0}{SE_d} = \frac{-4.20 - 0}{0.3208} = -13.09$

Step 4: Make the Decision

• Degrees of freedom: $df = n - 1 = 10 - 1 = 9$
• Critical value from $t$-table for one-tailed lower test at $\alpha = 0.05$ with $df = 9$: $t_{\text{crit}} = -1.833$
• Since $t_{\text{calc}} = -13.09 < -1.833$ (software $p < 0.0001$), we Reject $H_0$.
• Conclusion: The new ergonomic fixture achieves an extraordinary, statistically significant cycle time reduction of $4.20\text{ seconds}$ per harness ($p < 0.0001$).

---

When Parametric Assumptions Fail: The Non-Parametric Domain

Parametric tests ($t$-tests, ANOVA) rely strictly on the assumption that underlying continuous distributions are approximately Gaussian normal. In Six Sigma, three common conditions violate this assumption:

1. Severe Skewness / Asymmetry: Customer wait times, repair cycle times, and pollutant concentrations often exhibit extreme right-skewed lognormal or exponential shapes.
2. Small Sample Sizes with Outliers ($n < 30$): When sample sizes are small, the Central Limit Theorem cannot be invoked to ensure normality of sample means, and extreme outliers severely distort $\bar{x}$ and $s$.
3. Ordinal / Ranked Data: Customer satisfaction ratings (1 to 5 stars), pain scales, or supplier audit tier ranks are ordinal attributes, not true continuous interval metrics.

The Remediation Hierarchy

When data violates normality, Green Belts follow a structured three-step escalation protocol:

1. Check for Data Errors: Investigate whether extreme outliers are typos or measurement errors.
2. Attempt Mathematical Transformation: Apply Box-Cox or logarithmic power transformations to convert non-normal data into approximate normality.
3. Deploy Non-Parametric Tests: If data resists transformation or sample sizes are small, execute non-parametric tests.

---

Non-Parametric Tests for Medians & Ranks

Non-parametric tests—frequently termed distribution-free tests—make zero assumptions regarding the specific mathematical shape of the population. Instead of evaluating means and variances, non-parametric tests evaluate medians ($\tilde{\mu}$ or $\eta$) and analyze the ranks of ordered observations.

                   Parametric vs. Non-Parametric Test Mapping

   PARAMETRIC TEST (Means)                    NON-PARAMETRIC EQUIVALENT (Medians)
  ┌─────────────────────────────────────┐    ┌─────────────────────────────────────┐
  │ One-Sample t-Test                   │ ──▶│ 1-Sample Wilcoxon / 1-Sample Sign   │
  ├─────────────────────────────────────┤    ├─────────────────────────────────────┤
  │ Two-Sample Independent t-Test       │ ──▶│ Mann-Whitney U / Wilcoxon Rank-Sum  │
  ├─────────────────────────────────────┤    ├─────────────────────────────────────┤
  │ Paired t-Test                       │ ──▶│ Wilcoxon Signed-Rank Test           │
  ├─────────────────────────────────────┤    ├─────────────────────────────────────┤
  │ One-Way ANOVA                       │ ──▶│ Kruskal-Wallis / Mood's Median      │
  ├─────────────────────────────────────┤    ├─────────────────────────────────────┤
  │ Pearson Correlation (r)             │ ──▶│ Spearman's Rank Correlation (ρ)     │
  └─────────────────────────────────────┘    └─────────────────────────────────────┘

1. The Mann-Whitney U Test (Wilcoxon Rank-Sum Test)

• Parametric Equivalent: Two-Sample Independent $t$-test.
• Objective: Compares the medians of two independent, unrelated groups ($n_1$ and $n_2$).
• Mechanics: All observations from both groups are pooled into a single combined dataset and ranked in ascending order from $1$ to $N = n_1 + n_2$. Tied values receive average ranks. The sum of ranks for each group is calculated ($W_1$ and $W_2$). If the two populations are identical, the average rank in both groups should be equal.
• Test Statistic ($U$): $U_1 = n_1 n_2 + \frac{n_1(n_1 + 1)}{2} - W_1$ $U_2 = n_1 n_2 + \frac{n_2(n_2 + 1)}{2} - W_2$
• Hypotheses: $H_0: \tilde{\mu}_1 = \tilde{\mu}_2$ vs $H_a: \tilde{\mu}_1 \ne \tilde{\mu}_2$.

2. The Wilcoxon Signed-Rank Test (Paired)

• Parametric Equivalent: Paired $t$-test.
• Objective: Compares dependent, matched pairs when difference scores are non-normal but roughly symmetric.
• Mechanics: Differences $d_i = x_{1i} - x_{2i}$ are calculated. Any pairs where $d_i = 0$ are discarded. The absolute values $|d_i|$ are ranked from smallest to largest. The original positive or negative sign is reattached to each rank. The test sums the positive ranks ($W^+$) and negative ranks ($W^-$) and evaluates whether their sum deviates significantly from expected random variation.

3. The 1-Sample Sign Test

• Parametric Equivalent: One-Sample $t$-test.
• Objective: Tests whether a population median equals a specified target ($\tilde{\mu} = M_0$).
• Mechanics: Extremely simple: counts how many observations fall strictly above ($+$) and below ($-$) the hypothesized target $M_0$. Under $H_0$, the probability of an observation falling above the median is $p = 0.50$. The test uses the Binomial distribution to compute exact $p$-values. Highly robust against extreme outliers.

4. Kruskal-Wallis Test & Mood's Median Test

• Parametric Equivalent: One-Way ANOVA.
• Kruskal-Wallis: Compares medians across $k \ge 3$ independent groups by ranking all combined observations across groups. Sensitive to both shifts in median and general distribution shape.
• Mood's Median Test: Tests whether $k \ge 3$ groups share a common median by determining the grand median across all data, constructing a $2 \times k$ contingency table (counts above vs. below grand median), and running a Chi-Square test. Mood's median test is vastly more resistant to extreme outliers than Kruskal-Wallis, though it possesses slightly lower statistical power.

---

Parametric vs. Non-Parametric Selection Tradeoffs

The Six Sigma Rule of Thumb: If data is normal, always use parametric tests because they maximize statistical power. If data is severely non-normal, resists transformation, or consists of ordinal categories, deploy non-parametric tests to prevent invalid conclusions.

---

Critical CSSC Exam Traps

• Trap 1: Running an Independent Two-Sample Test on Paired Data — If you run an independent Two-Sample $t$-test on Before/After measurements, the test treats the $2n$ data points as unrelated, failing to subtract out unit-to-unit variation. The standard error balloons, causing a Type II Error (failing to detect a real improvement).
• Trap 2: Degrees of Freedom in Paired Studies — For a paired test with 12 pairs measured twice (24 total observations), $df = n_{\text{pairs}} - 1 = 12 - 1 = 11$, NOT $24 - 2 = 22$.
• Trap 3: Believing Non-Parametric Tests Evaluate Means — Non-parametric tests evaluate medians or rank sums, never arithmetic means.
• Trap 4: Mood's Median vs. Kruskal-Wallis Selection — Kruskal-Wallis is preferred for general non-normal data across $\ge 3$ groups. However, if extreme outliers or unequal subgroup shapes are present, Mood's Median test is the superior choice because it is completely unaffected by outlier magnitudes.