10.3 Hypothesis Testing Concepts: Hypotheses, Errors & P-Values

Hypothesis Testing Concepts: Hypotheses, Errors & P-Values

Quick Answer: Hypothesis testing provides an objective statistical method to determine whether an observed difference between samples represents a genuine process change or mere random sampling fluctuation. The null hypothesis ($H_0$) assumes no difference, while the alternative hypothesis ($H_a$) claims a real effect exists. Teams balance two risks: Type I error ($\alpha$ / Producer's Risk, false positive) and Type II error ($\beta$ / Consumer's Risk, false negative). The decision rule compares the calculated $p$-value to $\alpha$ (typically $0.05$): if $p < \alpha$, reject $H_0$ and conclude the difference is statistically significant. Independent CSSYB study guide by OpenExamPrep.

Principles of Statistical Hypothesis Testing

Six Sigma projects continually evaluate comparative questions: Did a tooling upgrade improve machining tolerances? Does Shift 1 produce fewer defects than Shift 2? Did a cycle time reduction initiative truly compress turnaround time, or was the sample just lucky?

Because all processes exhibit natural common-cause variation, sample means ($\bar{X}_1$ and $\bar{X}_2$) will rarely be identical even when processes are operating identically. If Shift 1 averages $14.2$ minutes and Shift 2 averages $14.8$ minutes, practitioners cannot rely on visual inspection to determine whether that $0.6$-minute gap is an operational shift or random noise.

Statistical hypothesis testing provides an objective, data-driven framework to resolve this dilemma. It uses probability theory to quantify the likelihood that an observed sample difference occurred purely by random chance.

---

Formulating Hypotheses: Null ($H_0$) vs. Alternative ($H_a$)

Every hypothesis test establishes two mutually exclusive and exhaustive statements about the population:

1. The Null Hypothesis ($H_0$)

The null hypothesis ($H_0$) represents the baseline assumption of no difference, no effect, or status quo ($\mu_1 = \mu_2$). It asserts that observed sample variations are entirely due to random sampling noise.

• The null hypothesis always contains an equality sign ($=$, $\le$, or $\ge$).
• In Six Sigma, $H_0$ is assumed true until sample data provides compelling statistical evidence to reject it (analogous to the presumption of innocence in a court of law).

2. The Alternative Hypothesis ($H_a$ or $H_1$)

The alternative hypothesis ($H_a$) represents the operational claim or improvement hypothesis being investigated. It asserts that a real difference, shift, or effect exists in the population.

• The alternative hypothesis never contains an equality sign ($\ne$, $>$, or $<$).
• Two-Tailed (Non-directional): $H_a: \mu_1 \ne \mu_2$ (detects any difference in either direction).
• One-Tailed (Directional): $H_a: \mu_1 > \mu_2$ or $H_a: \mu_1 < \mu_2$ (detects a difference in a specified direction).

---

The Decision Matrix and Error Types

When evaluating sample data, an improvement team decides to either Reject $H_0$ or Fail to Reject $H_0$. Comparing this decision to the unknown true state of the process yields a $2 \times 2$ decision matrix:

1. Type I Error ($\alpha$ / Producer's Risk)

A Type I error occurs when a team rejects a true null hypothesis (concluding a difference exists when there is none).

• Significance Level ($\alpha$): The maximum tolerable probability of committing a Type I error, typically set at $\alpha = 0.05$ ($5\%$).
• Operational Impact: Erroneously rejecting good product lots (Producer's Risk), leading to unnecessary scrap, re-testing, or unwarranted machine adjustments.

2. Type II Error ($\beta$ / Consumer's Risk)

A Type II error occurs when a team fails to reject a false null hypothesis (failing to detect a genuine difference).

• Symbol: Denoted by $\beta$.
• Operational Impact: Shipping defective parts to a customer because sample testing failed to detect a process shift (Consumer's Risk), causing field failures and warranty claims.

3. Statistical Power ($1 - \beta$)

Statistical power ($1 - \beta$) is the probability of correctly rejecting a false null hypothesis (detecting a real effect). Six Sigma studies target a minimum power of $0.80$ ($80\%$). Power is increased by enlarging sample size ($n$), detecting larger effect sizes ($\Delta$), or reducing process variability ($\sigma$).

---

The P-Value and Universal Decision Rule

Modern quality software computes a test statistic and its associated $p$-value.

What is a P-Value?

The $p$-value is the exact probability of obtaining a test result at least as extreme as the observed sample data, assuming the null hypothesis is true.

• A small $p$-value means the observed sample result is highly unlikely under $H_0$.
• A large $p$-value means the sample variation is consistent with common-cause noise.

The Universal Decision Rule

Practitioners compare the $p$-value directly to the significance level ($\alpha = 0.05$):

$$\text{If } p\text{-value} < \alpha \implies \textbf{Reject } H_0 \quad (\text{Statistically Significant Difference})$$ $$\text{If } p\text{-value} \ge \alpha \implies \textbf{Fail to Reject } H_0 \quad (\text{Insufficient Evidence of Difference})$$

Yellow Belt Rule of Thumb:

"If $p$ is low ($p < 0.05$), the null must go (Reject $H_0$)."
"If $p$ is high ($p \ge 0.05$), the null will fly (Fail to Reject $H_0$)."

Statistical vs. Practical Significance

A statistically significant result ($p < 0.05$) does not always guarantee practical or economic value. With very large sample sizes, a trivial difference (such as a $0.002\text{ mm}$ change) will achieve statistical significance, yet offer zero operational benefit.

---

Overview of Common Yellow Belt Hypothesis Tests

Six Sigma practitioners select tests based on data type and group structure: