7.8 Tests for Means, Variances, and Proportions

Comparative hypothesis testing evaluates process parameter differences across two or more experimental conditions, treatments, shifts, or operational lines. Choosing the correct parametric or non-parametric test depends on data type (continuous vs. discrete), sample independence (paired vs. independent), and variance homogeneity.

---

Two-Sample Hypothesis Tests for Means and Proportions

1. Independent Two-Sample $t$-Test

Compares the means of two independent continuous populations ($\mu_1$ vs. $\mu_2$).

Case A: Equal Variances ($\sigma_1^2 = \sigma_2^2$) — Pooled $t$-Test

• Pooled Variance ($s_p^2$): $$s_p^2 = \frac{(n_1 - 1) s_1^2 + (n_2 - 1) s_2^2}{n_1 + n_2 - 2}$$
• Test Statistic: $$t_0 = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{s_p^2 \left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}$$
• Degrees of Freedom: $\text{df} = n_1 + n_2 - 2$.

Case B: Unequal Variances ($\sigma_1^2 \neq \sigma_2^2$) — Welch's $t$-Test

Uses unpooled standard errors with Welch-Satterthwaite degrees of freedom approximation ($\nu$):

$$\nu = \frac{\left(\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}\right)^2}{\frac{(s_1^2/n_1)^2}{n_1 - 1} + \frac{(s_2^2/n_2)^2}{n_2 - 1}}$$

2. Paired Two-Sample $t$-Test

Used when observations are naturally paired or matched (e.g., before-and-after measurements on the exact same production parts, tools, or operators).

• Difference Variable: $d_i = x_{1i} - x_{2i}$
• Test Statistic: $$t_0 = \frac{\bar{d} - \mu_d}{\frac{s_d}{\sqrt{n}}}$$
• Degrees of Freedom: $\text{df} = n - 1$ (where $n$ is the number of matched pairs).
• Operational Advantage: Eliminates unit-to-unit baseline variability, greatly reducing residual error variance and maximizing test sensitivity/power.

3. Two-Sample Proportion $z$-Test

Compares two independent binomial success proportions ($p_1$ vs. $p_2$).

• Pooled Proportion ($\bar{p}$): $$\bar{p} = \frac{x_1 + x_2}{n_1 + n_2}$$
• Test Statistic: $$z_0 = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\bar{p}(1-\bar{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}$$

4. $F$-Test for Equality of Two Variances

Evaluates whether two independent population variances are equal ($H_0: \sigma_1^2 = \sigma_2^2$).

• Test Statistic: $$F_0 = \frac{s_1^2}{s_2^2} \quad \text{(placing larger sample variance in numerator, } s_1^2 \ge s_2^2 \text{)}$$
• Degrees of Freedom: $\text{df}_1 = n_1 - 1$, $\text{df}_2 = n_2 - 1$.
• Sensitivity Note: The $F$-test is highly sensitive to non-normality. Levene's test or Bartlett's test are preferred robust alternatives.

---

Non-Parametric Hypothesis Tests

When continuous data violates normality and transformation is unsuccessful, non-parametric tests based on data ranks must be deployed:

---

Tests for Variances

The Body of Knowledge requires tests for means, variances, and proportions. Variance tests are the ones teams most often skip, and they matter for two reasons: variation reduction is frequently the project's actual objective, and equality of variance is an assumption of the two-sample t-test and of ANOVA.

One sample against a target variance

$$\chi^2 = \frac{(n-1)s^2}{\sigma_0^2}, \qquad df = n - 1$$

Compare against the chi-square distribution. This test is very sensitive to non-normality and should not be used on visibly skewed data.

Two variances: the F test

$$F = \frac{s_1^2}{s_2^2}, \qquad df_1 = n_1 - 1,\ df_2 = n_2 - 1$$

By convention the larger variance goes in the numerator so $F \ge 1$, and the result is compared with $F_{\alpha/2, df_1, df_2}$ for a two-sided test.

Worked example. Line A: $n_1 = 16$, $s_1 = 2.4$. Line B: $n_2 = 13$, $s_2 = 1.5$.

$$F = \frac{2.4^2}{1.5^2} = \frac{5.76}{2.25} = 2.56$$

with 15 and 12 degrees of freedom. The two-sided critical value $F_{0.025, 15, 12} \approx 3.18$. Since $2.56 < 3.18$, do not reject: the evidence does not establish a difference in variability at the 5% level, despite the sample standard deviations differing by 60%. Variance tests need substantial sample sizes.

Like the chi-square test, the F test for variances is highly sensitive to departures from normality, much more so than the t-test is for means.

Several variances

Because Bartlett's test frequently rejects for non-normality rather than for unequal variance, Levene's test is the practical default when checking the equal-variance assumption before ANOVA.

When variances are unequal

Why variance tests matter to a Black Belt

A project that reduces the mean of a cycle time while leaving the spread unchanged has improved the average experience and left the worst case where it was. Customers experience the tail, not the mean. In many projects the correct primary metric is the standard deviation or a high percentile, and the corresponding hypothesis test is a variance test rather than a t-test.

The same logic drives the capability indices: $C_p$ is entirely a statement about variance, and a project targeting $C_p$ rather than $C_{pk}$ is targeting variation rather than centring.