10.4 Analyzing Experimental Results: Main Effects & Interaction Plots

10.4 Analyzing Experimental Results: Main Effects & Interaction Plots

Quick Summary: Once a designed experiment has been executed and response data collected, continuous improvement teams must extract actionable engineering insights. In Six Sigma, analyzing factorial experiments involves computing Main Effects (the individual impact of shifting a factor from Low to High) and Interaction Effects (the joint synergistic or antagonistic behavior between factors). These effects are visualized through Main Effects Plots and Interaction Plots. Practitioners must adhere to the cardinal Hierarchy of Effects Rule: when a significant interaction exists, main effects cannot be evaluated in isolation. By integrating Factorial ANOVA tables, diagnostic residual auditing, and mathematical transfer functions, Green Belts establish optimal operating windows that permanently resolve process problems.

---

The Objective of Experimental Data Analysis

Executing a $2^k$ factorial experiment generates an orthogonal matrix of paired factor settings and empirical response measurements. Data analysis answers four vital operational questions in the DMAIC Analyze and Improve phases:

1. Which input factors ($X$) exert a statistically significant effect on output CTQ ($Y$)?
2. Do the factors interact with one another? That is, does the impact of Factor A depend on the operating level of Factor B?
3. What is the mathematical direction and magnitude of each effect?
4. What exact combination of factor settings optimizes the response variable while minimizing process variability?

                  The DOE Analytical Architecture

   RAW RESPONSE DATA FROM RANDOMIZED TRIALS
   ┌─────────────────────────────────────────────────────────┐
   │ Runs 1 through N: Measured CTQ Response Values (Y)      │
   └───────────────────────────┬─────────────────────────────┘
                               ▼
   ORTHOGONAL CONTRAST CALCULATIONS
   ┌─────────────────────────────────────────────────────────┐
   │ Compute Main Effects: Effect(A), Effect(B), Effect(C)   │
   │ Compute Interaction Effects: Effect(AB), Effect(BC)...  │
   └───────────────────────────┬─────────────────────────────┘
                               ▼
   GRAPHICAL & STATISTICAL EVALUATION
   ┌───────────────────────────┴─────────────────────────────┐
   ▼                                                         ▼
   VISUAL EFFECTS PLOTS                               FACTORIAL ANOVA TABLE
   • Main Effects Plots (Slope & Magnitude)           • SS, MS, F-ratios, p-values
   • Interaction Plots (Parallel vs. Crossing)        • LINE Residual Diagnostics
   └───────────────────────────┬─────────────────────────────┘
                               ▼
   EMPIRICAL PREDICTION MODEL & OPTIMIZATION
   ┌─────────────────────────────────────────────────────────┐
   │ ŷ = b0 + b1(x1) + b2(x2) + b12(x1*x2) ──▶ OPTIMAL RECIPE │
   └─────────────────────────────────────────────────────────┘

---

Mechanics of Calculating Main Effects

Formal Mathematical Definition

A Main Effect measures the average change in response $Y$ when an independent factor shifts from its Low level ($-1$) to its High level ($+1$), averaged across all levels of all other experimental factors:

$\text{Main Effect}_A = \bar{Y}_{A(+)} - \bar{Y}_{A(-)}$

Where:

• $\bar{Y}_{A(+)}$ is the arithmetic mean of all experimental observations where Factor A was set to High ($+1$).
• $\bar{Y}_{A(-)}$ is the arithmetic mean of all experimental observations where Factor A was set to Low ($-1$).

Interpreting the Sign and Magnitude

• Positive Main Effect ($+15.0$): Shifting Factor A from Low to High increases the average process response by $15.0$ measurement units.
• Negative Main Effect ($-8.5$): Shifting Factor A from Low to High decreases the average process response by $8.5$ measurement units.
• Zero Main Effect ($0.0$): Factor A has no linear effect on the response; changing its setting does not shift the mean of $Y$.

Connection to Regression Coefficients in Coded Units

In a $2^k$ factorial model, factor levels are coded as $-1$ and $+1$. The total distance between Low and High along the coded axis is: $\Delta x = (+1) - (-1) = 2\text{ coded units}$

Therefore, the linear regression slope coefficient ($b_1$) in coded units is exactly one-half of the main effect:

$b_1 = \frac{\text{Main Effect}_A}{2}$

This is a classic CSSC Green Belt examination question: if the main effect of cutting speed is $+12.0\text{ mm/s}$, the corresponding regression coefficient $b_1$ in the coded model $\hat{y} = b_0 + b_1 x$ is $+6.0$.

---

Mechanics of Calculating Interaction Effects

Formal Mathematical Definition

An Interaction Effect occurs when the effect that Factor A exerts on the response depends directly on the operating level of Factor B:

In mathematical terms, factors interact when their joint effect is not equal to the sum of their individual main effects: $\text{Effect}(AB) \ne \text{Effect}(A) + \text{Effect}(B)$.

In a 2-factor design, the interaction effect is calculated as half the difference between the effect of Factor A at the High level of B and the effect of Factor A at the Low level of B:

$\text{Interaction Effect}_{AB} = \frac{(\bar{Y}_{A+B+} - \bar{Y}_{A-B+}) - (\bar{Y}_{A+B-} - \bar{Y}_{A-B-})}{2} = \bar{Y}_{AB(+)} - \bar{Y}_{AB(-)}$

Where:

• $\bar{Y}_{AB(+)}$ is the mean of all runs where the coded product column $A \times B$ equals $+1$ (Runs where A and B are both Low, or both High).
• $\bar{Y}_{AB(-)}$ is the mean of all runs where the coded product column $A \times B$ equals $-1$ (Runs where one factor is High and the other is Low).

Synergistic vs. Antagonistic Interactions

• Synergistic Interaction: Factors enhance each other's performance. The combined effect of operating both factors at High is far greater than the sum of their separate individual effects.
• Antagonistic (Interfering) Interaction: Factors counteract or cancel each other out. Increasing Factor A produces a positive benefit when B is Low, but causes catastrophic degradation when B is High.

---

Visual Interpretation of Effects Plots

Continuous improvement practitioners communicate experimental outcomes using two standardized graphics: Main Effects Plots and Interaction Plots.

                    Interpreting Effects Plots in DOE

      MAIN EFFECTS PLOT                    INTERACTION PLOT: PARALLEL LINES
   Response Y ▲                           Response Y ▲
              │            ○ (+1)                    │         ○─────────────○ B (+1)
              │          ╱                           │        ╱             ╱
              │        ╱                             │       ╱             ╱
              │      ╱                               │      ○─────────────○ B (-1)
              │    ○ (-1)                            │     -1 (Low)      +1 (High)
              └────┴───────────────►                 └─────┴─────────────┴─────► Factor A
                  -1 (Low)        +1 (High)          Parallel Lines = NO INTERACTION
                  Steep Slope = Strong Effect        (Effect of A is identical at all levels of B)

      INTERACTION PLOT: MODERATE NON-PARALLEL  INTERACTION PLOT: SEVERE CROSSING ('X')
   Response Y ▲                           Response Y ▲
              │               ○ B (+1)               │         ○ B (+1)      ○ B (-1)
              │             ╱                        │           ╲         ╱
              │           ╱                          │             ╲     ╱
              │   ○─────○ B (-1)                     │               ╳   INTERSECTION!
              │                                      │             ╱     ╲
              └───┴───────────┴────────►             │           ╱         ╲
                 -1 (Low)    +1 (High)               └─────────○ B (-1)──────○ B (+1)► Factor A
                 Non-Parallel = Moderate Interaction  Severe Crossing = ANTAGONISTIC INTERACTION!

Diagnostic Reading of the Plots

1. Main Effects Plot:

   • Horizontal (Flat) Line: The factor has zero effect on the response. Changing its setting produces no measurable change in $Y$.
   • Steep Upward Slope: Factor has a strong positive main effect. Increasing the factor increases $Y$.
   • Steep Downward Slope: Factor has a strong negative main effect. Increasing the factor decreases $Y$.

2. Interaction Plot:

   • Parallel Lines: Zero Interaction. The line representing Factor B at $+1$ is completely parallel to the line representing Factor B at $-1$. The effect of Factor A is completely identical regardless of the setting of Factor B. Main effects can be interpreted independently.
   • Non-Parallel (Diverging or Converging) Lines: Moderate Interaction. The effect of Factor A is stronger at one level of B than at the other. Main effects are somewhat dependent.
   • Intersecting / Crossing Lines ('X' Pattern): Severe / Disordinal Interaction. The lines cross each other directly! Increasing Factor A increases $Y$ when B is High, but decreases $Y$ when B is Low. The effect completely reverses direction!

The Hierarchy of Effects Principle (Interaction Precedence Rule)

The Cardinal Law of DOE: When an interaction effect between two factors is statistically significant, you cannot interpret or optimize the main effects in isolation!

If an interaction plot displays crossing lines, a Green Belt who looks only at the Main Effects Plot will see a flat horizontal line (the positive effect at High B and negative effect at Low B cancel each other out, averaging to zero). The Green Belt would mistakenly conclude that neither factor matters! In reality, both factors exert massive control over the process, but their impact depends entirely on their joint combination. The interaction term always takes precedence.

---

Factorial ANOVA: Partitioning Sums of Squares

To prove whether observed main and interaction effects are statistically significant or merely artifacts of background process noise, practitioners conduct an Analysis of Variance (ANOVA).

Partitioning Total Variation

In a 2-factor full factorial experiment with $n$ replicates ($N = n \cdot 2^2 = 4n$ total runs), the total sum of squares ($SS_{total}$) is partitioned into four orthogonal, additive components:

$SS_{total} = SS_A + SS_B + SS_{AB} + SS_{error}$

Calculating Sums of Squares for Effects

Because the design matrix is perfectly orthogonal, the sum of squares for any factor or interaction term in a $2^k$ design with $N$ total runs is calculated directly from its effect magnitude:

$SS_{\text{Effect}} = \frac{N}{4} \cdot (\text{Effect})^2 = \frac{(\text{Contrast})^2}{N}$

Where $\text{Contrast} = \sum_{i=1}^N c_i y_i$, with $c_i \in {-1, +1}$.

Degrees of Freedom and Mean Squares

• Each main effect and two-way interaction in a 2-level design has exactly $1$ degree of freedom ($DF = 1$).
• Therefore, the Mean Square is identical to the Sum of Squares: $MS_{\text{Effect}} = SS_{\text{Effect}} / 1 = SS_{\text{Effect}}$.
• The Error Sum of Squares ($SS_{error}$) represents pure experimental error within replicate treatment combinations, with $DF_{error} = 4(n - 1)$.
• Mean Square Error: $MS_{error} = SS_{error} / DF_{error}$.

The $F$-Test Decision Rule

For each experimental term, calculate the $F$-statistic:

$F_{\text{Effect}} = \frac{MS_{\text{Effect}}}{MS_{error}}$

• If $p \le 0.05$ ($F > F_{\text{crit}}$), Reject $H_0$ and conclude that the effect is statistically significant.
• If $p > 0.05$, the effect is indistinguishable from random experimental noise.

---

Residual Diagnostics for Experimental Models

Before accepting the ANOVA conclusions, Green Belts must verify the underlying assumptions of the linear model using residual diagnostic plots ($e_i = y_i - \hat{y}_i$):

1. Normal Probability Plot of Residuals: Verifies that experimental errors follow a Gaussian normal distribution. Points must hug the diagonal line ($p > 0.05$ on Anderson-Darling test).
2. Residuals versus Fitted Values: Checks for homoscedasticity (constant variance across treatment combinations). Residual dispersion must remain constant across all predicted values; a funnel or megaphone shape indicates variance instability.
3. Residuals versus Run Order: Verifies the assumption of independence. Residuals plotted against chronological execution order must show random scatter. Trends, drifts, or runs indicate that lurking variables (e.g., machine warm-up, tool wear) contaminated the experiment due to failed randomization.

---

Developing the Empirical Transfer Function ($Y = f(X)$)

Once significant terms are identified via ANOVA, practitioners construct the predictive empirical model using coded coefficients:

$\hat{y} = b_0 + b_1 x_A + b_2 x_B + b_{12} x_A x_B$

Where:

• $b_0 = \bar{y}$ is the overall grand mean of all experimental runs,
• $b_1 = \text{Effect}(A) / 2$,
• $b_2 = \text{Effect}(B) / 2$,
• $b_{12} = \text{Effect}(AB) / 2$,
• $x_A, x_B \in {-1, +1}$ are factor settings in standardized coded units.

This transfer function allows the Six Sigma team to predict process performance for any operating setting within the design space and identify the exact parameter combination that maximizes product quality.

---

Step-by-Step Comprehensive Worked Calculation Example: Replicated $2^2$ Design

Problem: An ultrasonic welding process for medical syringe barrels experiences variable weld shear strength ($Y$, in Newtons). A Six Sigma Green Belt conducts a $2^2$ full factorial experiment with $n = 2$ replicates ($N = 8$ total runs) testing two factors:

• Factor A (Weld Time): Low $= 0.6\text{ s}$ ($-1$), High $= 1.2\text{ s}$ ($+1$)
• Factor B (Clamp Pressure): Low $= 30\text{ psi}$ ($-1$), High $= 60\text{ psi}$ ($+1$)

The randomized trials yield the following shear strength results:

Calculate the Main Effect of A, Main Effect of B, Interaction Effect AB, complete the Factorial ANOVA Table, determine statistical significance at $\alpha = 0.05$, and formulate the predictive transfer function.

Step 1: Compute Main Effect of Factor A (Weld Time)

• Observations where $A = +1$: Runs 2 and 4 (4 total observations) $\bar{Y}_{A(+)} = \frac{120.0 + 176.0}{4} = \frac{296.0}{4} = 74.0\text{ N}$
• Observations where $A = -1$: Runs 1 and 3 (4 total observations) $\bar{Y}_{A(-)} = \frac{88.0 + 104.0}{4} = \frac{192.0}{4} = 48.0\text{ N}$
• Main Effect of A: $\text{Effect}(A) = \bar{Y}_{A(+)} - \bar{Y}_{A(-)} = 74.0 - 48.0 = \mathbf{+26.0\text{ N}}$

Step 2: Compute Main Effect of Factor B (Clamp Pressure)

• Observations where $B = +1$: Runs 3 and 4 (4 total observations) $\bar{Y}_{B(+)} = \frac{104.0 + 176.0}{4} = \frac{280.0}{4} = 70.0\text{ N}$
• Observations where $B = -1$: Runs 1 and 2 (4 total observations) $\bar{Y}_{B(-)} = \frac{88.0 + 120.0}{4} = \frac{208.0}{4} = 52.0\text{ N}$
• Main Effect of B: $\text{Effect}(B) = \bar{Y}_{B(+)} - \bar{Y}_{B(-)} = 70.0 - 52.0 = \mathbf{+18.0\text{ N}}$

Step 3: Compute Interaction Effect AB (Time $\times$ Pressure)

• Observations where $AB = +1$: Runs 1 and 4 (4 total observations) $\bar{Y}_{AB(+)} = \frac{88.0 + 176.0}{4} = \frac{264.0}{4} = 66.0\text{ N}$
• Observations where $AB = -1$: Runs 2 and 3 (4 total observations) $\bar{Y}_{AB(-)} = \frac{120.0 + 104.0}{4} = \frac{224.0}{4} = 56.0\text{ N}$
• Interaction Effect of AB: $\text{Effect}(AB) = \bar{Y}_{AB(+)} - \bar{Y}_{AB(-)} = 66.0 - 56.0 = \mathbf{+10.0\text{ N}}$

Step 4: Compute Sums of Squares ($SS$)

Using $N = 8$ total runs and $SS_{\text{Effect}} = \frac{N}{4} (\text{Effect})^2 = 2 \times (\text{Effect})^2$:

• $SS_A = 2 \times (26.0)^2 = 2 \times 676 = \mathbf{1352.0}$
• $SS_B = 2 \times (18.0)^2 = 2 \times 324 = \mathbf{648.0}$
• $SS_{AB} = 2 \times (10.0)^2 = 2 \times 100 = \mathbf{200.0}$

Compute Pure Error Sum of Squares ($SS_{error}$): Within each of the 4 treatment combinations, compute squared differences between replicate pairs ($d_j = y_{j1} - y_{j2}$), where $SS_{error} = \sum d_j^2 / 2$:

• Run 1: $(42 - 46)^2 = (-4)^2 = 16$
• Run 2: $(58 - 62)^2 = (-4)^2 = 16$
• Run 3: $(50 - 54)^2 = (-4)^2 = 16$
• Run 4: $(86 - 90)^2 = (-4)^2 = 16$ $SS_{error} = \frac{16 + 16 + 16 + 16}{2} = \frac{64}{2} = \mathbf{32.0}$

Compute Total Sum of Squares ($SS_{total}$): $SS_{total} = SS_A + SS_B + SS_{AB} + SS_{error} = 1352.0 + 648.0 + 200.0 + 32.0 = \mathbf{2232.0}$

Step 5: Complete the Factorial ANOVA Table

Statistical Decision at $\alpha = 0.05$:

• Critical $F$-value for $DF = (1, 4)$ at $\alpha = 0.05$ is $F_{0.05, 1, 4} = 7.71$.
• Since $F_A = 169.00 > 7.71$, Factor A is statistically significant ($p < 0.001$).
• Since $F_B = 81.00 > 7.71$, Factor B is statistically significant ($p < 0.001$).
• Since $F_{AB} = 25.00 > 7.71$, the Interaction AB is statistically significant ($p = 0.007$).

Step 6: Construct the Empirical Transfer Function

• Grand mean: $\bar{y} = G / N = 488.0 / 8 = 61.0\text{ N}$
• Coded regression coefficients:
  • $b_0 = 61.0$
  • $b_1 = \text{Effect}(A) / 2 = 26.0 / 2 = +13.0$
  • $b_2 = \text{Effect}(B) / 2 = 18.0 / 2 = +9.0$
  • $b_{12} = \text{Effect}(AB) / 2 = 10.0 / 2 = +5.0$

$\hat{y} = 61.0 + 13.0 x_A + 9.0 x_B + 5.0 x_A x_B$

Step 7: Practical Interpretation and Optimal Settings

Notice the synergy in the interaction term ($+5.0 x_A x_B$):

• When Clamp Pressure is Low ($x_B = -1$), the effect of Weld Time is: $(13.0 - 5.0) \times 2 = 16.0\text{ N}$ ($60 - 44 = 16\text{ N}$).
• When Clamp Pressure is High ($x_B = +1$), the effect of Weld Time accelerates to: $(13.0 + 5.0) \times 2 = 36.0\text{ N}$ ($88 - 52 = 36\text{ N}$).
• Because weld shear strength must be maximized, the optimal operating recipe is to operate both factors at High ($x_A = +1, x_B = +1$): $\hat{y} = 61.0 + 13.0(1) + 9.0(1) + 5.0(1)(1) = 61.0 + 27.0 = \mathbf{88.0\text{ N}}$ Operating at $1.2\text{ s}$ weld time and $60\text{ psi}$ clamp pressure delivers the maximum robust weld strength.

---

Critical Exam Traps to Avoid

• Trap 1: Evaluating Main Effects When Interaction is Statistically Significant — If an interaction plot displays non-parallel or crossing lines and $p_{AB} \le 0.05$, exam questions will tempt candidates to pick factor settings based on the main effects plot alone. The interaction term always takes precedence.
• Trap 2: Forgetting to Divide Effect by Two for Coded Regression Coefficients — Remember that coded levels span 2 units (from $-1$ to $+1$). Therefore, $b_i = \text{Effect}_i / 2$. Confusing the full effect with the slope coefficient is a frequent calculation blunder.
• Trap 3: Assuming Parallel Lines on an Interaction Plot Indicate Severe Interaction — Parallel lines indicate zero interaction. Lines must diverge, converge, or cross to indicate an interaction.
• Trap 4: Conflating Pure Error Degrees of Freedom with Total Sample Size — In replicated factorial designs, $DF_{error} = 2^k (n - 1)$. For an un-replicated design ($n = 1$), $DF_{error} = 0$, meaning pure error cannot be estimated without pooling higher-order interactions.