7.2 Hypothesis Testing: Means, Proportions, and Variances

Hypothesis testing is the formalized mathematical procedure by which industrial and systems engineers make objective, data-driven decisions regarding processes, machinery, materials, and human workflows. Rather than relying on subjective intuition or raw numerical differences that could simply be artifacts of random sampling variability, hypothesis testing quantifies the probability that an observed effect is genuine. The NCEES FE Reference Handbook provides the complete inventory of test statistics under the Engineering Probability and Statistics section. Understanding how to set up hypotheses, avoid decision errors, calculate test statistics, and interpret p-values is essential for the FE examination.

---

1. The Hypothesis Testing Framework

Null ($H_0$) and Alternative ($H_1$ or $H_a$) Hypotheses

Every statistical test establishes two mutually exclusive and exhaustive propositions:

1. Null Hypothesis ($H_0$): Represents the baseline status quo, default assumption, historical standard, or claim of no effect or no difference. The null hypothesis must always contain the condition of equality ($=$, $\le$, or $\ge$).
2. Alternative Hypothesis ($H_1$): Represents the experimental claim, departure from standard, new operating condition, or research hypothesis to be proven ($<$, $>$, or $\neq$).

Directionality: Two-Tailed vs. One-Tailed Tests

   Two-Tailed (α/2 in each tail)       One-Tailed Upper (α in right tail)
         ┌───────┴───────┐                           ┌───┴───┐
        ┌─┐             ┌─┐                         │       ┌─┐
     ───┴─┴─────────────┴─┴───                   ───┴───────┴─┴───
     -Z_crit           +Z_crit                             +Z_crit

---

2. Decision Errors, Risks, and Test Power

Because sample statistics are random variables, hypothesis testing inherently involves uncertainty. The two potential incorrect decisions define Type I and Type II errors.

Industrial Engineering Analogs: Producer's vs. Consumer's Risk

• Type I Error ($\alpha$, Producer's Risk): Rejecting a good batch of parts or stopping a production process that is actually functioning within specification. The manufacturer ("producer") incurs the unnecessary cost of scrap, sorting, or downtime.
• Type II Error ($\beta$, Consumer's Risk): Failing to reject a defective batch of parts or allowing an out-of-control machine to continue operating. The customer ("consumer") receives non-conforming product, leading to field failures, warranty claims, or safety hazards.

Statistical Power ($1 - \beta$)

Power is the probability of correctly rejecting a false null hypothesis: $\text{Power} = P(\text{Reject } H_0 \mid H_0 \text{ is False}) = 1 - \beta$. Power can be increased by:

1. Increasing sample size $n$: Reduces standard error, narrowing distributions and decreasing overlap.
2. Increasing significance level $\alpha$: Moving the critical threshold inwards decreases $\beta$ (at the cost of increasing Type I risk).
3. Larger effect size ($\delta = |\mu - \mu_0|$): Substantial real differences are easier to detect.
4. Decreasing process variance ($\sigma^2$): Improving process capability tightens the distributions.

---

3. Decision Rules: Critical Value Approach vs. p-Value Approach

The Critical Value Method

Compute the test statistic from the sample data ($Z_0, t_0, F_0, \chi_0^2$). If the test statistic falls into the designated rejection region defined by the critical value at significance level $\alpha$, reject $H_0$; otherwise, fail to reject $H_0$.

The p-Value Method

The $p$-value is the probability, assuming $H_0$ is true, of obtaining a test statistic at least as extreme as the observed value.

$$\text{Golden Decision Rule:} \quad \begin{cases} p\text{-value} \le \alpha & \implies \text{Reject } H_0 \text{ (Statistically Significant)} \\ p\text{-value} > \alpha & \implies \text{Fail to Reject } H_0 \end{cases}$$

• For a right-tailed $Z$-test: $p\text{-value} = P(Z \ge Z_0)$.
• For a left-tailed $Z$-test: $p\text{-value} = P(Z \le Z_0)$.
• For a two-tailed $Z$-test: $p\text{-value} = 2 \times P(Z \ge |Z_0|)$.

---

4. Test Statistics for a Single Population Mean (μ)

Case 1: Variance Known (Z-Test)

When $\sigma$ is known and either the population is normal or $n \ge 30$:

$$Z_0 = \frac{\bar{x} - \mu_0}{\sigma / \sqrt{n}}$$

Case 2: Variance Unknown (One-Sample t-Test)

When $\sigma$ is unknown, estimate with $s$:

$$t_0 = \frac{\bar{x} - \mu_0}{s / \sqrt{n}}$$

with degrees of freedom $\nu = n - 1$. Compare $t_0$ against critical values $t_{\alpha, \nu}$ or $t_{\alpha/2, \nu}$.

---

5. Two-Sample Tests for Comparing Population Means

Independent Samples with Equal Variances (Pooled t-Test)

Testing $H_0: \mu_1 - \mu_2 = \Delta_0$ (usually $\Delta_0 = 0$) under the assumption $\sigma_1^2 = \sigma_2^2$:

$$t_0 = \frac{(\bar{x}_1 - \bar{x}_2) - \Delta_0}{s_p \sqrt{\frac{1}{n_1} + \frac{1}{n_2}}}, \quad \text{where } s_p = \sqrt{\frac{(n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + n_2 - 2}}$$

Degrees of freedom: $\nu = n_1 + n_2 - 2$.

Independent Samples with Unequal Variances (Welch's t-Test)

When $\sigma_1^2 \neq \sigma_2^2$:

$$t_0 = \frac{(\bar{x}_1 - \bar{x}_2) - \Delta_0}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}$$

with Satterthwaite degrees of freedom $\nu$.

Paired Samples (Paired t-Test)

For paired observations with differences $d_i = x_{1i} - x_{2i}$, testing $H_0: \mu_d = \Delta_{d0}$:

$$t_0 = \frac{\bar{d} - \Delta_{d0}}{s_d / \sqrt{n}}$$

with degrees of freedom $\nu = n - 1$, where $\bar{d}$ is the sample mean of differences and $s_d$ is the sample standard deviation of differences.

---

6. Hypothesis Testing for Proportions

Single-Sample Proportion Z-Test

Testing $H_0: p = p_0$ versus $H_1: p \neq p_0$ (or $p > p_0$, $p < p_0$):

$$Z_0 = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1 - p_0)}{n}}}$$

Exam Watchout: Standard Error in Single Proportion Tests Under the null hypothesis, the true proportion is assumed to equal $p_0$. Therefore, the standard error in the denominator must be evaluated using $p_0$, not $\hat{p}$. In contrast, confidence intervals use $\hat{p}$ because no hypothesized value exists. Substituting $\hat{p}$ into the denominator of a hypothesis test is a classic distractor on the FE exam.

Two-Sample Proportion Z-Test

Testing $H_0: p_1 - p_2 = 0$ against $H_1: p_1 \neq p_2$. Because the null hypothesis posits that $p_1 = p_2 = p$, data from both samples are pooled to form the best single estimate of the common proportion $\bar{p}$:

$$\bar{p} = \frac{x_1 + x_2}{n_1 + n_2}$$

The test statistic is:

$$Z_0 = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\bar{p}(1 - \bar{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}$$

---

7. Tests on Variances: Chi-Square and F-Tests

Single Population Variance (Chi-Square Test)

Testing $H_0: \sigma^2 = \sigma_0^2$ (or $\sigma^2 \le \sigma_0^2$, $\sigma^2 \ge \sigma_0^2$):

$$\chi_0^2 = \frac{(n - 1)s^2}{\sigma_0^2}$$

with degrees of freedom $\nu = n - 1$. Compare $\chi_0^2$ against critical values $\chi^2_{\alpha, \nu}$ or $\chi^2_{\alpha/2, \nu}$.

Two Population Variances (Snedecor's F-Test)

Testing equality of variances between two independent normal populations: $H_0: \sigma_1^2 = \sigma_2^2$ versus $H_1: \sigma_1^2 \neq \sigma_2^2$ (or $\sigma_1^2 > \sigma_2^2$).

The test statistic is the ratio of sample variances:

$$F_0 = \frac{s_1^2}{s_2^2}$$

with numerator degrees of freedom $\nu_1 = n_1 - 1$ and denominator degrees of freedom $\nu_2 = n_2 - 1$.

Standard Calculation Convention: Always designate the sample with the larger sample variance as Sample 1, so that $s_1^2 \ge s_2^2$ and $F_0 \ge 1$. For a two-tailed test at significance level $\alpha$, compare $F_0$ to the upper-tail critical value $F_{\alpha/2, \, \nu_1, \, \nu_2}$. If $F_0 > F_{\alpha/2, \, \nu_1, \, \nu_2}$, reject $H_0$ and conclude the variances are unequal.

---

8. Practical vs. Statistical Significance in Industrial Manufacturing

In high-volume manufacturing environments equipped with automated sensing (e.g., in-line vision inspection logging $n = 100{,}000$ parts per day), standard error $\sigma / \sqrt{n}$ approaches zero. As a result, even microscopic, trivial deviations—such as a shaft measuring $25.0002\text{ mm}$ versus a $25.0000\text{ mm}$ target—will yield an astronomical test statistic ($Z_0 > 10$) and a tiny $p$-value ($p < 0.00001$).

Industrial engineers must distinguish between:

• Statistical Significance: The observed difference is unlikely to be explained by chance alone ($p \le \alpha$).
• Practical (Engineering) Significance: The observed difference is large enough to impact product function, assembly fit, customer satisfaction, or unit economics. A change of $0.0002\text{ mm}$ on a part with a $\pm 0.100\text{ mm}$ tolerance band is statistically significant but practically meaningless.

---

9. Step-by-Step Worked Engineering Examples

Worked Example 7.2.1: Single-Sample t-Test on Polymer Tensile Strength

Problem: An additive manufacturing facility prints tooling fixtures. The nominal tensile yield strength must meet or exceed $\mu_0 = 45.0\text{ MPa}$. A random sample of $n = 16$ test dogbones yields a sample mean of $\bar{x} = 43.8\text{ MPa}$ and a sample standard deviation of $s = 2.0\text{ MPa}$. Test at significance level $\alpha = 0.05$ whether the fixtures fail to meet specification.

Solution:

1. Formulate Hypotheses:
   • $H_0: \mu \ge 45.0\text{ MPa}$ (compliant tensile strength)
   • $H_1: \mu < 45.0\text{ MPa}$ (undersized / non-compliant tensile strength, lower one-tailed test)
2. Select Test Statistic and Identify Degrees of Freedom:
   • Since $\sigma$ is unknown, use the one-sample $t$-test with $\nu = n - 1 = 16 - 1 = 15$.
3. Compute the Test Statistic: $$t_0 = \frac{\bar{x} - \mu_0}{s / \sqrt{n}} = \frac{43.8 - 45.0}{2.0 / \sqrt{16}} = \frac{-1.2}{2.0 / 4} = \frac{-1.2}{0.50} = -2.40$$
4. Determine Critical Value and Rejection Region:
   • Significance level: $\alpha = 0.05$.
   • Degrees of freedom: $\nu = 15$.
   • For a lower one-tailed test, the critical value is $-t_{0.05, 15} = -1.753$.
   • Rejection region: Reject $H_0$ if $t_0 < -1.753$.
5. Statistical Decision and Engineering Interpretation:
   • Since $t_0 = -2.40 < -1.753$, we reject $H_0$ at $\alpha = 0.05$.
   • Conclusion: There is statistically significant evidence at the $5\%$ level that the 3D-printed polymer fixtures fail to meet the required $45.0\text{ MPa}$ tensile specification. The production batch must be quarantined.

Worked Example 7.2.2: Two-Sample Proportion Test on Solder Bridging

Problem: A surface-mount electronics line experiences solder bridging defects. An engineer tests a new nitrogen-inerting reflow process against the baseline air process:

• Baseline Air ($1$): $x_1 = 36$ defectives out of $n_1 = 400$ boards ($\hat{p}_1 = 0.090$).
• Nitrogen Inerting ($2$): $x_2 = 18$ defectives out of $n_2 = 400$ boards ($\hat{p}_2 = 0.045$). At $\alpha = 0.05$, has the nitrogen process significantly reduced the defect rate?

Solution:

1. Formulate Hypotheses:
   • $H_0: p_1 - p_2 \le 0$ (nitrogen does not reduce defects)
   • $H_1: p_1 - p_2 > 0$ (nitrogen significantly reduces defects, right-tailed test)
2. Compute Pooled Proportion $\bar{p}$: $$\bar{p} = \frac{x_1 + x_2}{n_1 + n_2} = \frac{36 + 18}{400 + 400} = \frac{54}{800} = 0.0675$$ $$1 - \bar{p} = 1 - 0.0675 = 0.9325$$
3. Compute Pooled Standard Error: $$\text{SE} = \sqrt{\bar{p}(1 - \bar{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)} = \sqrt{0.0675 \times 0.9325 \times \left(\frac{1}{400} + \frac{1}{400}\right)}$$ $$\text{SE} = \sqrt{0.062944 \times 0.0050} = \sqrt{0.00031472} \approx 0.01774$$
4. Compute the Test Statistic $Z_0$: $$Z_0 = \frac{\hat{p}_1 - \hat{p}_2}{\text{SE}} = \frac{0.090 - 0.045}{0.01774} = \frac{0.045}{0.01774} \approx +2.537$$
5. Decision and Conclusion:
   • For an upper one-tailed test at $\alpha = 0.05$, critical value $Z_{0.05} = 1.645$.
   • Because $Z_0 = 2.537 > 1.645$ (corresponding $p\text{-value} \approx 0.0056 < 0.05$), we reject $H_0$.
   • Conclusion: The nitrogen reflow process significantly reduces the solder bridge defect rate from $9.0\%$ to $4.5\%$.

---

10. NCEES Reference Handbook Tips & Realistic Exam Traps

• Equality Belongs in the Null: On conceptual setup questions, look for the choice where the equals sign ($=, \le, \ge$) is in $H_0$. Any choice placing an inequality without an equals sign ($<, >, \neq$) in $H_0$ is automatically invalid.
• Two-Tailed p-Value Doubling: When given a test statistic $Z_0 = 2.10$ for a two-tailed test, finding $P(Z \ge 2.10) = 0.0179$ from standard normal tables is only half the answer. You must double it for a two-tailed test: $p\text{-value} = 2 \times 0.0179 = 0.0358$.
• F-Test Degree of Freedom Ordering: When looking up $F_{\alpha, \nu_1, \nu_2}$, make sure $\nu_1$ is the numerator degrees of freedom (columns in the NCEES handbook table) and $\nu_2$ is the denominator degrees of freedom (rows). Reversing them results in a completely wrong critical value because the $F$-distribution is not symmetric.
• Pooled Proportion in Two-Sample Z-Tests: Never use unpooled sample variances when testing $H_0: p_1 = p_2$. The standard error must use $\bar{p} = (x_1 + x_2)/(n_1 + n_2)$.