9.4 Analysis of Variance (One-Way & Two-Way ANOVA) & Chi-Square Contingency Tests

9.4 Analysis of Variance (One-Way & Two-Way ANOVA) & Chi-Square Contingency Tests

Quick Summary: When continuous improvement investigations expand beyond two groups or involve categorical attribute variables, practitioners deploy advanced inferential tools. One-Way Analysis of Variance (ANOVA) compares continuous means across three or more groups ($k \ge 3$) simultaneously using the $F$-ratio ($MS_{\text{Between}} / MS_{\text{Within}}$), preventing the massive Type I error inflation caused by multiple pairwise $t$-tests. When two factors influence the process, Two-Way ANOVA evaluates both main effects and their interaction effect. For categorical and discrete defect counts, the Chi-Square ($\chi^2$) Test of Independence evaluates contingency tables to detect associations between process attributes.

---

One-Way Analysis of Variance (ANOVA)

Purpose & Operational Context

One-Way ANOVA (Analysis of Variance) is a parametric hypothesis test used to compare the means of three or more independent groups ($k \ge 3$) on a single continuous response variable ($Y$). Examples include:

• Comparing tensile strength across four raw material resin suppliers.
• Comparing order cycle time across three operating shifts (Day, Evening, Night).
• Comparing dimensional accuracy across five CNC machining centers.

Hypotheses Formulation

• $H_0: \mu_1 = \mu_2 = \mu_3 = \dots = \mu_k$ (All group population means are equal)
• $H_a:$ At least one group mean is different from the others

Crucial Exam Rule: The alternative hypothesis in ANOVA is NOT "all means are different." $H_a$ simply states that at least one group mean is statistically distinguishable from at least one other group mean. In fact, three groups could have identical means while a fourth group is radically higher, satisfying $H_a$.

Why Multiple $t$-Tests Are Strictly Prohibited: $\alpha$-Inflation

A frequent novice mistake in Six Sigma is comparing four groups by running separate two-sample $t$-tests on every pair (Group 1 vs 2, 1 vs 3, 1 vs 4, 2 vs 3, 2 vs 4, 3 vs 4).

The number of pairwise comparisons ($c$) required for $k$ groups is:

$$c = \frac{k(k - 1)}{2}$$

If each individual test is conducted at significance level $\alpha = 0.05$, the family-wise Type I error rate (the overall probability of committing at least one false alarm across the family of tests) inflates exponentially:

$$\alpha_{\text{family}} = 1 - (1 - \alpha)^c$$

For 5 groups, running separate $t$-tests produces a $40.1\%$ chance of falsely claiming a significant difference when none exists! ANOVA solves this problem by conducting a single, omnibus test that evaluates all means simultaneously while strictly holding $\alpha = 0.05$.

---

The Mechanics of the F-Statistic & The ANOVA Table

ANOVA evaluates differences in means by analyzing and partitioning variances. The total variation in the data is split into two independent components:

                         Partitioning Total Variance

                              TOTAL VARIATION
                                 (SS_Total)
                                     │
                    ┌────────────────┴────────────────┐
                    ▼                                 ▼
          BETWEEN-GROUP VARIATION            WITHIN-GROUP VARIATION
               (SS_Between)                       (SS_Within)
           Variation caused by               Random, unexplained
            Factor Treatment                 background noise / error

1. Between-Group Variance ($SS_{\text{Between}}$ or $SS_{\text{Factor}}$): Measures how far the individual group sample means deviate from the overall grand mean ($\bar{\bar{x}}$). This represents the signal generated by process differences.
2. Within-Group Variance ($SS_{\text{Within}}$ or $SS_{\text{Error}}$): Measures how individual observations deviate from their respective group sample mean. This represents inherent, unexplained random noise.

Mean Squares & The $F$-Ratio

Dividing Sum of Squares ($SS$) by their respective Degrees of Freedom ($df$) yields Mean Squares ($MS$):

$$F_{\text{calc}} = \frac{MS_{\text{Between}}}{MS_{\text{Within}}} = \frac{\text{Between-Group Variance (Signal)}}{\text{Within-Group Variance (Noise)}}$$

• If $F \approx 1$: Between-group differences are no larger than random process noise ($p > 0.05$; Fail to reject $H_0$).
• If $F \gg 1$: The variance between group averages is substantially larger than random background noise ($p \le 0.05$; Reject $H_0$).

The Standard ANOVA Source Table

(Where $k = $ number of groups, $N = $ total observations across all groups).

Step-by-Step Worked Example: Interpreting an ANOVA Table

Scenario: A Green Belt compares the yield strength of metal brackets across $k = 4$ independent heat-treatment recipes. A total of $N = 24$ brackets were tested ($n = 6$ brackets per recipe). The partial ANOVA table shows:

• $SS_{\text{Factor}} = 180.0$
• $SS_{\text{Error}} = 120.0$

Calculate Degrees of Freedom, Mean Squares, and $F$-Statistic:

1. $df_{\text{Factor}} = k - 1 = 4 - 1 = 3$
2. $df_{\text{Error}} = N - k = 24 - 4 = 20$
3. $df_{\text{Total}} = N - 1 = 24 - 1 = 23$
4. $MS_{\text{Factor}} = \frac{180.0}{3} = 60.0$
5. $MS_{\text{Error}} = \frac{120.0}{20} = 6.0$
6. $F_{\text{calc}} = \frac{MS_{\text{Factor}}}{MS_{\text{Error}}} = \frac{60.0}{6.0} = 10.00$

At $\alpha = 0.05$, the critical $F_{0.05, 3, 20} = 3.10$. Because $F_{\text{calc}} = 10.00 > 3.10$ ($p < 0.001$), we Reject $H_0$. Heat-treatment recipe significantly affects yield strength.

---

Post-Hoc Analysis: Tukey's Honestly Significant Difference (HSD)

A statistically significant ANOVA test is an omnibus test: it proves that a difference exists somewhere among the means, but it does not identify which specific groups differ.

To identify which groups differ without inflating Type I error, practitioners run a post-hoc multiple comparison test. The gold standard in Six Sigma is Tukey's Honestly Significant Difference (HSD) test.

Interpreting Tukey's Grouping Output

Statistical software groups means using letters. Groups that do not share a common letter are statistically significantly different:

                  Tukey HSD Output Interpretation

      Recipe      Sample Size     Mean Yield      Grouping
     ──────────────────────────────────────────────────────
      Recipe C        6            142.5 MPa         A
      Recipe A        6            138.0 MPa         A
      Recipe D        6            124.2 MPa            B
      Recipe B        6            112.8 MPa               C

• Recipes C and A share letter A: No significant difference between C and A.
• Recipe D has letter B: Recipe D is significantly lower than A and C, but significantly higher than B.
• Recipe B has letter C: Recipe B is significantly lower than all other recipes.

---

Two-Way ANOVA: Main Effects & Interactions

While One-Way ANOVA examines a single independent factor, industrial processes are usually driven by multiple variables acting simultaneously. Two-Way ANOVA evaluates two independent factors (Factor $A$ and Factor $B$) on a continuous response $Y$, testing three distinct hypotheses:

1. Main Effect of Factor $A$: Does Factor $A$ significantly alter $Y$, averaged across all levels of Factor $B$?
2. Main Effect of Factor $B$: Does Factor $B$ significantly alter $Y$, averaged across all levels of Factor $A$?
3. Interaction Effect ($A \times B$): Does the effect of Factor $A$ depend upon the specific level of Factor $B$?

Interpreting Interaction Plots

Interaction plots graph the response mean on the vertical axis against Factor $A$ on the horizontal axis, with separate lines for each level of Factor $B$:

                     Interpreting Interaction Plots

      NO INTERACTION (Parallel Lines)           STRONG INTERACTION (Crossing Lines)
   Y ┌                     B2 (High)         Y ┌               B1 (High)    B2
     │                ───────                  │                  ╲        ╱
     │               ╱                         │                   ╲      ╱
     │              ╱                          │                    ╲    ╱
     │             ╱   B1 (Low)                │                     ╲  ╱
     │       ───────                           │                      ╳
     │      ╱                                  │                     ╱  ╲
     │     ╱                                   │                    ╱    ╲
     └────────────────────────                 └─────────────────────────────
       A1 (Low)       A2 (High)                  A1 (Low)            A2 (High)
       Effect of A is same across B              Effect of A REVERSES across B!

• Parallel Lines: Zero interaction. The effect of Factor $A$ is consistent regardless of Factor $B$. The factors act independently.
• Non-Parallel or Crossing Lines: Strong interaction present. The effect of Factor $A$ depends entirely on what level Factor $B$ is set to. When an interaction is significant, main effects cannot be interpreted in isolation.

---

Chi-Square ($\chi^2$) Tests for Categorical Data

When data is categorical or discrete attribute counts (e.g., defect type, region, supplier, pass/fail), ANOVA cannot be applied. Practitioners deploy Chi-Square ($\chi^2$) tests.

1. Chi-Square Test of Independence (Contingency Tables)

Evaluates whether two categorical variables are statistically independent or whether an association exists between them (e.g., Defect Category vs. Manufacturing Shift).

Data is tabulated in an $r \times c$ contingency table ($r$ rows, $c$ columns):

$$\chi^2_{\text{calc}} = \sum_{i=1}^r \sum_{j=1}^c \frac{(O_{ij} - E_{ij})^2}{E_{ij}}$$

Where:

• $O_{ij} = $ Observed frequency count in row $i$, column $j$
• $E_{ij} = $ Expected frequency count under the null hypothesis of independence: $$E_{ij} = \frac{(\text{Row } i \text{ Total}) \times (\text{Column } j \text{ Total})}{\text{Grand Total } N}$$
• $\mathbf{Degrees\ of\ Freedom:}\ df = (r - 1)(c - 1)$

The Expected Cell Frequency Rule: The asymptotic Chi-Square distribution is valid only if all expected cell frequencies satisfy $E_{ij} \ge 5$. If any expected cell count is $< 5$, categories must be combined, or Fisher's Exact Test must be used for $2 \times 2$ tables.

Step-by-Step Worked Example: Chi-Square Contingency Test

Scenario: A Green Belt investigates whether defect classification (Scratch vs Dent) depends on production shift (Day vs Night). A sample of $N = 200$ defective brackets yields the following observed counts ($O$):

Test for independence at $\alpha = 0.05$.

Step 1: Compute Expected Frequencies ($E_{ij}$)

• $E_{\text{Day, Scratch}} = \frac{100 \times 90}{200} = 45.0$
• $E_{\text{Day, Dent}} = \frac{100 \times 110}{200} = 55.0$
• $E_{\text{Night, Scratch}} = \frac{100 \times 90}{200} = 45.0$
• $E_{\text{Night, Dent}} = \frac{100 \times 110}{200} = 55.0$ (All $E_{ij} \ge 5$, satisfying the cell frequency rule).

Step 2: Calculate Chi-Square Statistic

$$\chi^2_{\text{calc}} = \frac{(60 - 45)^2}{45} + \frac{(40 - 55)^2}{55} + \frac{(30 - 45)^2}{45} + \frac{(70 - 55)^2}{55}$$ $$\chi^2_{\text{calc}} = \frac{225}{45} + \frac{225}{55} + \frac{225}{45} + \frac{225}{55} = 5.000 + 4.091 + 5.000 + 4.091 = 18.182$$

Step 3: Determine Degrees of Freedom and Decision

• $df = (r - 1)(c - 1) = (2 - 1)(2 - 1) = 1 \times 1 = 1$
• Critical value at $\alpha = 0.05$ with $df = 1$: $\chi^2_{\text{crit}} = 3.841$
• Since $\chi^2_{\text{calc}} = 18.182 > 3.841$ ($p < 0.0001$), Reject $H_0$.
• Conclusion: Defect type is strongly associated with shift; Night Shift produces significantly more dents, while Day Shift produces significantly more scratches.

---

Master Inferential Hypothesis Testing Roadmap

Use this comprehensive master roadmap to select the exact hypothesis test required for any Six Sigma examination scenario:

---

Critical CSSC Exam Traps

• Trap 1: Running Multiple Pairwise $t$-Tests Instead of ANOVA — This severely inflates the family-wise Type I error rate ($\alpha$-inflation), creating massive false alarm risks.
• Trap 2: Believing $H_a$ in ANOVA Means "All Means Differ" — The alternative hypothesis in ANOVA only requires that at least one group mean differs from the others.
• Trap 3: Calculating Degrees of Freedom for Contingency Tables as $N - 1$ — For a Chi-Square test of independence on an $r \times c$ table, degrees of freedom is strictly $df = (r - 1)(c - 1)$.
• Trap 4: Ignoring the Expected Cell Frequency Requirement — Chi-Square tests are invalid if any expected cell count is $< 5$. Always verify $E_{ij} = (\text{Row Total} \times \text{Column Total}) / N \ge 5$.
• Trap 5: Interpreting Main Effects When Interaction is Significant — In Two-Way ANOVA, if the interaction $p$-value $\le 0.05$, you must examine the interaction plot before drawing conclusions about main factors.