Hypothesis Testing Basics

Hypothesis Testing Basics

Quick Answer: Hypothesis testing is a structured decision method: state H₀ and Hₐ, pick α (Type I risk), collect a valid sample, compute a test statistic and p-value, then reject H₀ if p ≤ α (or the statistic falls in the critical region)—while still asking whether the effect is practically large enough to justify Improve actions.

Why Green Belts Use Hypothesis Tests

In Analyze, sponsors ask: Is line A really different from line B? Did the pilot change cycle time? A hypothesis test answers with a transparent error framework instead of gut feel on one snapshot. CSSGB BoK IV.B.1 is at the Apply level: set up the test correctly, interpret α, power, Type I/II errors, sample size, and statistical-vs-practical significance—not derive every distribution from first principles.

Null, Alternative, and Decision Language

Decision outcomes

1. Reject H₀ — sample evidence is strong enough (relative to α) against the null.
2. Fail to reject H₀ — evidence is not strong enough. This is not proof that H₀ is true; the effect may be small, noise large, or n too small (low power).

Never write “accept H₀ as proven.” Exam language favors reject vs fail to reject.

One-sided vs two-sided Hₐ

• Two-sided: “different from” (μ ≠ μ₀)—common default when either direction matters.
• One-sided: “greater than” or “less than” only when the process story makes the opposite direction irrelevant before seeing the data. Do not flip to one-sided after peeking at results.

Significance Level (α), p-Value, and the Decision Rule

α (significance level) is chosen in advance. It is the long-run Type I error rate you tolerate when H₀ is true. Problems often use α = 0.05; high-cost false alarms may use 0.01; exploratory screening may use 0.10.

p-value (conceptually): probability of a test statistic at least as extreme as observed if H₀ were true. Small p-values mean the data are surprising under H₀.

Rule used throughout CSSGB practice

• If p ≤ α → reject H₀ (statistically significant at level α).
• If p > α → fail to reject H₀.

What p is not: not the probability H₀ is true, not the probability Hₐ is true, and not the size of the improvement. A p of 0.04 does not mean a 4% defect reduction.

Type I Error, Type II Error, and Power

Trade-offs:

• Lower α (stricter Type I control) generally raises β unless you increase sample size.
• Power rises when the true effect is larger, variation is smaller, n is larger, or α is larger.
• Plan n so power is adequate for the smallest effect worth finding, not only for convenient sample sizes.

α vs β risk example: A supplier change is expensive. A false alarm (Type I) may force unnecessary requalification; missing a true quality drop (Type II) ships defects. Choose α with stakeholders, then plan n for the shift you care about.

Sample Size and Detectable Effects

For fixed α and assumed σ (or baseline proportion):

• Larger n → smaller standard error → higher power for a given true difference.
• Larger n can also make a tiny difference statistically significant—hence practical significance.
• Small n → often fail to reject even when a useful effect exists (underpowered study).

Green Belt practice: estimate σ from Measure, state the minimum mean or proportion difference that matters economically, pick α and target power (often ~0.80–0.90), then size n. If n is infeasible, reduce measurement noise (MSA) or accept lower power knowingly.

Statistical vs Practical Significance

Example: n = 5,000 fills; mean is 0.02 mL above target; p = 0.001. Statistically significant, but if specs allow ±2 mL and change cost is huge, practical impact may be negligible. Conversely, p = 0.08 with a large estimated yield gain and modest n may justify a better-powered confirmation—not automatic rejection of the idea.

Always report: decision on H₀, estimated effect size, and business recommendation.

Green Belt Decision Framework

1. Define the business question in Y and X language.
2. Write H₀ / Hₐ and one- vs two-sided before analysis.
3. Set α and, when planning, power and n.
4. Check assumptions (independence, normality/robustness, equal variance when required, data type).
5. Run the test; apply p ≤ α.
6. Translate into process action—never stop at “significant.”
7. Confirm important claims with control charts, capability, or DOE before full-scale Improve.

Pitfalls and Exam Focus

• Confusing “fail to reject” with “proved equal.”
• Treating α = 0.05 as a law rather than a risk choice.
• Chasing significance by retesting the same data many ways without care.
• Ignoring power when a pilot “shows no difference.”
• Equating a small p-value with a large or causal effect.

CSSGB items at IV.B.1 typically ask you to name Type I/II, link α to false alarms, connect power to sample size and effect size, and choose correctly when statistical and practical significance diverge.