8.1 Design of Experiments: Objectives, Terminology, and Effects
Key Takeaways
- Design of Experiments (DOE) is a structured statistical approach that systematically manipulates input factors (X's) to evaluate their direct main effects and interactive impacts on critical output responses (Y's).
- The three fundamental principles of experimental design are Randomization (protects against lurking variable bias and time-order trends), Replication (enables calculation of pure experimental error variance σ²), and Blocking (isolates known nuisance sources of variation).
- Main Effect of a factor measures the average change in response produced by changing a factor from its low (-1) level to its high (+1) level: E_A = Y_bar_{A+} - Y_bar_{A-}.
- An Interaction Effect occurs when the impact of one factor on the response variable depends on the specific level setting of another factor, indicated graphically by non-parallel or intersecting lines.
- DOE follows a structured, sequential strategy progressing from screening designs (identifying active factors) to characterization factorials, optimization (RSM), and Taguchi robust designs.
DOE Objectives and Strategic Framework
In the Six Sigma Improve Phase, Design of Experiments (DOE) is the definitive statistical methodology used to optimize process parameters, establish cause-and-effect relationships between process inputs ($X$'s) and key outputs ($Y$'s), and uncover complex factor interactions.
Traditional engineering approaches rely on One-Factor-at-a-Time (OFAT) experimentation, where an experimenter changes a single factor while holding all others constant. OFAT is fundamentally flawed because:
- It cannot detect interaction effects between input factors.
- It requires significantly more experimental runs to achieve equivalent statistical power.
- It often leads to false local optima.
Sequential DOE Strategy
DOE is executed sequentially across four strategic stages:
- Screening: Identifying the few critical vital factors ($X$'s) out of many potential inputs using Resolution III or IV fractional factorials.
- Characterization: Quantifying main effects and two-factor interactions using $2^k$ full factorials or Resolution V fractional factorials.
- Optimization: Modeling quadratic non-linear curves to locate optimal operational settings using Response Surface Methodology (RSM).
- Robustness Testing: Ensuring process settings remain immune to uncontrollable environmental or raw material noise factors using Taguchi designs.
Key DOE Terminology
Understanding standardized DOE terminology is essential for the CSSBB exam:
| Term | Definition & Context | Practical Process Example |
|---|---|---|
| Factor ($X$) | An independent process variable directly manipulated by the experimenter. | Temperature, Pressure, Concentration. |
| Level | A specific setting or value assigned to a factor during an experiment. | Temperature Low (-1) = 150°C, High (+1) = 200°C. |
| Treatment | A specific combination of factor levels tested in a single experimental run. | Run with Temp = 200°C (+1) and Pressure = 50 psi (-1). |
| Run | A single unit of execution or observation under specified treatment settings. | Performing one weld cycle under defined settings. |
| Response ($Y$) | The measured dependent outcome variable evaluating process performance. | Weld Shear Strength (psi), Yield Rate (%). |
| Experimental Unit | The smallest physical entity to which a treatment is independently applied. | A single silicon wafer, a discrete steel coupon. |
| Covariate | An uncontrolled, continuous variable measured during the experiment to adjust for nuisance effects statistically. | Ambient relative humidity during adhesive curing. |
| Noise Factor | An uncontrollable or expensive-to-control variable causing response variation. | Shift operator technique, raw material batch variation. |
| Crossed factors | Every level of one factor is run with every level of the other, which is what makes an interaction estimable. | Each of 3 temperatures run on each of 2 machines. |
| Nested factor | A factor whose levels are unique to a single level of another factor rather than shared across them. | Operators 1-3 work only line A and operators 4-6 only line B; batches within a supplier. |
Nesting is a design property, not a choice of analysis. Because operator 1 never appears on line B, there is no operator-by-line cell to compare, so the interaction simply does not exist and cannot be estimated. A nested (hierarchical) design therefore reports variance components -- how much of the total variation lives between lines, between operators within a line, and within an operator -- rather than main effects and interactions. Analyzing nested data as though it were crossed is a standard exam trap; the tell is that a factor's levels carry different meanings under each level of the factor above it. Gage R&R is the familiar case: parts crossed with operators when every operator measures every part, nested when they cannot (destructive testing).
Calculating Main Effects and Interaction Effects
For a standard two-level $2^2$ factorial design evaluating Factors $A$ and $B$, four treatment combinations exist: $(1)$, $a$, $b$, and $ab$, where lowercase letters represent the high (+1) level of that factor:
| Run | Factor A | Factor B | Coded AB Interaction | Response ($Y$) |
|---|---|---|---|---|
| 1 | -1 | -1 | +1 | $Y_{(1)}$ |
| 2 | +1 | -1 | -1 | $Y_{a}$ |
| 3 | -1 | +1 | -1 | $Y_{b}$ |
| 4 | +1 | +1 | +1 | $Y_{ab}$ |
Main Effect Calculation
The Main Effect of Factor $A$ ($E_A$) measures the average change in response when Factor $A$ moves from its low (-1) setting to its high (+1) setting:
Similarly, the Main Effect of Factor $B$ ($E_B$) is:
Interaction Effect Calculation
An Interaction Effect ($INT_{AB}$) measures the degree to which the effect of Factor $A$ depends on the level of Factor $B$. It is calculated as half the difference between the effect of Factor $A$ at the high level of $B$ and the effect of Factor $A$ at the low level of $B$:
Graphical Interpretation of Effects
- Main Effects Plot: Plots average response values at low and high levels. Steeper slopes indicate stronger main effects.
- Interaction Plot: Displays response curves of Factor $A$ across both levels of Factor $B$.
- Parallel Lines: Zero interaction between factors.
- Non-Parallel / Crossing Lines: Strong interaction present. When significant interaction exists, main effects cannot be interpreted independently.
Degrees of Freedom and Sum of Squares in $2^k$ Factorials
In a $2^k$ design with $n$ replicates (total runs $N_{total} = n \cdot 2^k$):
- Total Degrees of Freedom: $DF_{Total} = N_{total} - 1$
- Factor Degrees of Freedom: $DF_{Factor} = 1$ (for 2-level factors)
- Error Degrees of Freedom: $DF_{Error} = 2^k(n - 1)$
The Sum of Squares for any two-level factor effect ($SS_{Factor}$) is computed directly from its effect estimate:
In ANOVA analysis, the variance ratio test $F_0 = \frac{MS_{Factor}}{MS_{Error}}$ determines statistical significance ($p < 0.05$).
A $2^2$ factorial experiment yields the following average response values: Run (1) [A-, B-] = 10, Run a [A+, B-] = 24, Run b [A-, B+] = 14, and Run ab [A+, B+] = 38. What is the calculated Two-Factor Interaction Effect (INT_AB)?
An experimenter testing a chemical coating process applies treatment settings to four separate metal test coupons, resets temperature and pressure controls, and processes four new coupons under identical settings on a second day. What experimental principle is executed?
Why would a Black Belt incorporate Blocking into the design matrix of a multi-day industrial experiment?