12.1 Design of Experiments: 2^k Factorials, ANOVA & Taguchi Robust Design

12.1 Design of Experiments: 2^k Factorials, ANOVA & Taguchi Robust Design

Design of Experiments (DOE) is an active, structured empirical methodology for systematically manipulating process inputs (factors) and observing the resulting changes in system outputs (responses). Developed by Sir Ronald A. Fisher in agricultural research and subsequently adapted to industrial engineering by George Box, Genichi Taguchi, and Douglas Montgomery, DOE enables engineers to optimize processes, improve yield, reduce variability, and establish robust operating windows.

---

1. Experimental Design Fundamentals

In industrial systems, any manufacturing or service process can be modeled as a transfer function:

$y = f(x_1, x_2, \dots, x_k) + \epsilon$

where $y$ is the measured performance response, $x_i$ represent controllable or uncontrollable input factors, and $\epsilon$ represents random experimental error.

Core Terminology

• Factor: An independent variable manipulated by the experimenter. Factors can be quantitative (e.g., furnace temperature in °C, feed rate in mm/min) or qualitative/categorical (e.g., machine operator, material supplier, catalyst type).
• Level: The specific values or operational settings assigned to a factor during the experiment. In two-level designs, levels are typically designated as low ($-1$) and high ($+1$).
• Treatment (Treatment Combination): A specific set of factor levels applied to an experimental unit. In a design with $k$ factors, each unique combination of settings across all $k$ factors constitutes one treatment.
• Response: The dependent variable of interest measured after applying a treatment (e.g., tensile strength in MPa, surface roughness in $\mu$m, assembly cycle time in seconds).
• Experimental Error: The residual, unexplained variability among experimental units treated identically. It arises from unmeasured environmental fluctuations, material inconsistencies, and measurement system noise.

Fisher's Three Foundational Principles

To ensure valid statistical inference, any experimental plan must implement three core principles:

1. Replication: The independent repetition of the basic experiment (treatment combinations applied to distinct experimental units). True replication must be distinguished from repeated measurements: taking five readings on a single machined part reflects measurement error, whereas machining five distinct parts under identical nominal machine settings constitutes true replication. Replication achieves two vital objectives:
   • It allows the experimenter to obtain an independent, unbiased estimate of pure experimental error variance ($\sigma^2$), which forms the denominator of test statistics in Analysis of Variance (ANOVA).
   • It reduces the standard error of the sample mean: $\sigma_{\bar{y}} = \frac{\sigma}{\sqrt{n}}$, improving the precision of effect estimates.
2. Randomization: Both the allocation of experimental material and the run sequence of treatment combinations are determined by a random mechanism (e.g., pseudo-random number generation). Randomization:
   • Neutralizes the impact of extraneous, uncontrolled, or lurking variables (e.g., ambient temperature rise throughout a workday, tool wear, raw material aging).
   • Validates the statistical assumption that the error terms $\epsilon_{ij}$ are independently and identically distributed (i.i.d.) normal random variables: $\epsilon \sim \text{NID}(0, \sigma^2)$.
3. Blocking: A design technique used to isolate and eliminate the variability introduced by known, controllable nuisance factors. A block is a relatively homogeneous set of experimental units (e.g., a single operator shift, a specific batch of raw chemical reagent, or a single production day). Treatments are randomized within each block, allowing the block-to-block variability to be partitioned out of the residual experimental error in the ANOVA model.

---

2. One-Factor-at-a-Time (OFAT) vs. Full Factorial Designs

A historically pervasive yet statistically flawed experimental approach is the One-Factor-at-a-Time (OFAT) method. Under OFAT, an engineer fixes all process variables at arbitrary baseline values, varies factor $A$ across its range to locate an apparent optimum, holds factor $A$ at that new optimum, varies factor $B$, and repeats this sequentially.

The Failure Modes of OFAT

1. Inability to Detect Interactions: OFAT implicitly assumes that factors act independently (i.e., that the system is strictly additive: $f(x_1, x_2) = g(x_1) + h(x_2)$). When interactions exist, the effect of factor $A$ changes depending on the level of factor $B$. OFAT is mathematically blind to these interactions, often missing the true global optimum and converging on sub-optimal local peaks.
2. Statistical Inefficiency: To achieve comparable precision for main effects, OFAT requires significantly more experimental runs than a factorial design. In factorial designs, every single run provides information about every factor simultaneously—a property known as hidden replication.
3. Narrow Generalizability: Conclusions drawn from OFAT apply only under the specific, fixed levels chosen for the remaining factors. If baseline settings are changed, the observed relationships frequently collapse.

---

3. $2^k$ Factorial Designs: Matrix Architecture and Orthogonality

A $2^k$ Factorial Design investigates $k$ factors, each evaluated at exactly two levels: low (coded as $-1$) and high (coded as $+1$). A complete single replicate requires $N = 2^k$ experimental runs.

Coded Variable Transformation

Quantitative factors are centered and scaled from natural engineering units ($X_i$) to dimensionless coded variables ($x_i$) using the linear transformation:

$x_i = \frac{X_i - \left(\frac{X_{i,\text{high}} + X_{i,\text{low}}}{2}\right)}{\frac{X_{i,\text{high}} - X_{i,\text{low}}}{2}}$

Coding normalizes all factor scales to $[-1, +1]$, removing units of measurement and ensuring that the magnitudes of calculated regression coefficients directly reflect their relative physical importance.

Standard Run Order (Yates' Order) and Run Labels

Runs in a $2^k$ design are designated using standard notation where lowercase letters represent the factors present at their high ($+1$) level in that treatment combination. If all factors are at their low ($-1$) level, the treatment is denoted by the symbol $(1)$:

• $2^2$ Design (4 runs): $(1), a, b, ab$
• $2^3$ Design (8 runs): $(1), a, b, ab, c, ac, bc, abc$

Sign Table and Design Matrix for a $2^3$ Factorial

The columns for interaction effects are generated by algebraic point-by-point multiplication of the corresponding main factor columns:

The Mathematical Property of Orthogonality

The design matrix $\mathbf{X}$ possesses two vital mathematical properties:

1. Zero Sum: The sum of coefficients in any factor or interaction column (excluding identity $I$) is zero: $\sum_{i=1}^{2^k} c_i = 0$.
2. Orthogonal Dot Product: The inner product of any two distinct columns $\mathbf{c}_j$ and $\mathbf{c}_m$ ($j \neq m$) is identically zero:

$\mathbf{c}_j^T \mathbf{c}_m = \sum_{i=1}^{2^k} c_{ij} c_{im} = 0$

Because the matrix $\mathbf{X}^T \mathbf{X} = N \mathbf{I}$ is diagonal, all main effect and interaction estimates are statistically independent and uncorrelated. Adding or removing a factor from the analysis does not alter the numerical estimates of the remaining effects.

---

4. Main Effect, Contrast, and Sum of Squares Formulas

For a $2^k$ factorial experiment replicated $n$ times (total observations $N = n \cdot 2^k$):

1. Contrast

The contrast for any effect is the linear combination of treatment totals weighted by the sign vector ($c_i \in \{-1, +1\}$) from the sign table:

$\text{Contrast}_A = \sum_{i=1}^{2^k} c_i \cdot y_{i\cdot} = y_{A(+)} - y_{A(-)}$

where $y_{i\cdot}$ is the sum of all $n$ replicates for treatment combination $i$, and $y_{A(+)}$ and $y_{A(-)}$ represent the aggregate responses across all runs where factor $A$ is at its high and low levels, respectively.

2. Main Effect

The main effect represents the average change in response produced by moving factor $A$ from its low level to its high level:

$\text{Effect}_A = \frac{\text{Contrast}_A}{n \cdot 2^{k-1}} = \bar{y}_{A(+)} - \bar{y}_{A(-)}$

Note that the denominator $n \cdot 2^{k-1}$ is the total number of observations at the high level (which equals the total number at the low level).

3. Regression Coefficient

In the coded first-order regression model $y = \beta_0 + \sum \beta_i x_i + \sum \sum \beta_{ij} x_i x_j + \dots$, the regression coefficient is exactly half the effect estimate because the coded factor spans 2 units (from $-1$ to $+1$):

$\beta_A = \frac{\text{Effect}_A}{2} = \frac{\text{Contrast}_A}{n \cdot 2^k}$

4. Sum of Squares (SS)

The sum of squares attributable to any factor or interaction effect with 1 degree of freedom is:

$SS_A = \frac{(\text{Contrast}_A)^2}{n \cdot 2^k} = \frac{n \cdot 2^k}{4} (\text{Effect}_A)^2$

---

5. Two-Factor and Three-Factor Interactions

An interaction occurs when the effect of one factor is contingent upon the setting of another factor.

Mathematical Formulation of a 2-Factor Interaction

The interaction effect $AB$ is half the difference between the effect of $A$ at the high level of $B$ and the effect of $A$ at the low level of $B$:

$AB = \frac{1}{2} \left[ A_{(B=+1)} - A_{(B=-1)} \right] = \frac{\text{Contrast}_{AB}}{n \cdot 2^{k-1}}$

For a $2^2$ design:

$\text{Contrast}_{AB} = [ab - b] - [a - (1)] = ab - b - a + (1)$

Graphical Interpretation via Interaction Plots

Interaction plots display the response $y$ on the vertical axis against factor $A$ on the horizontal axis, with separate lines plotted for each level of factor $B$:

• Parallel Lines: Zero interaction. The effect of factor $A$ is identical regardless of factor $B$'s setting. The system is purely additive: $SS_{AB} \approx 0$.
• Non-Parallel, Non-Intersecting Lines (Ordinal / Synergistic Interaction): The magnitude of factor $A$'s effect changes depending on $B$, but the direction of the effect remains consistent. For instance, raising temperature always increases yield, but it increases yield far more dramatically when pressure is high.
• Intersecting Lines (Disordinal / Antagonistic Interaction): The sign of factor $A$'s effect completely reverses depending on the setting of factor $B$. For example, increasing cutting speed improves tool life at low feed rates, but severely degrades tool life at high feed rates. In the presence of a strong disordinal interaction, main effects have little physical meaning on their own and should never be interpreted in isolation.

---

6. Analysis of Variance (ANOVA) for $2^k$ Factorials

ANOVA provides the formal statistical framework for hypothesis testing in experimental design, decomposing total variability into orthogonal components.

Decomposition of Total Sum of Squares

$SS_{\text{Total}} = SS_{\text{Model}} + SS_{\text{Error}}$

$SS_{\text{Total}} = \sum_{i=1}^{2^k} \sum_{j=1}^n (y_{ij} - \bar{y}_{\cdot\cdot})^2 = \sum_{i=1}^{2^k} \sum_{j=1}^n y_{ij}^2 - \frac{Y_{\cdot\cdot}^2}{N}$

where $CF = \frac{Y_{\cdot\cdot}^2}{N}$ is the correction factor for the mean, $Y_{\cdot\cdot}$ is the grand total of all $N = n \cdot 2^k$ observations, and $\bar{y}_{\cdot\cdot} = \frac{Y_{\cdot\cdot}}{N}$ is the grand mean.

Because the factorial design matrix is orthogonal, $SS_{\text{Model}}$ is the exact linear sum of the sums of squares of all individual main effects and interaction effects:

$SS_{\text{Model}} = \sum_{j=1}^k SS_j + \sum_{j < m} SS_{jm} + \dots + SS_{12\dots k}$

Error Sum of Squares ($SS_{\text{Error}}$)

Pure experimental error is obtained by summing the internal variation within each of the $2^k$ treatment combinations:

$SS_{\text{Error}} = \sum_{i=1}^{2^k} \sum_{j=1}^n (y_{ij} - \bar{y}_{i\cdot})^2 = SS_{\text{Total}} - SS_{\text{Model}}$

Degrees of Freedom ($df$)

• Total degrees of freedom: $df_{\text{Total}} = N - 1 = n \cdot 2^k - 1$
• Each main effect or interaction: $df_{\text{Effect}} = 1$
• Model degrees of freedom: $df_{\text{Model}} = 2^k - 1$
• Error degrees of freedom: $df_{\text{Error}} = 2^k(n - 1) = N - 2^k$

Mean Squares and the $F$-Test Statistic

For each term, the Mean Square is its sum of squares divided by its degrees of freedom:

$MS_{\text{Effect}} = \frac{SS_{\text{Effect}}}{1} = SS_{\text{Effect}}$

$MS_{\text{Error}} = \frac{SS_{\text{Error}}}{df_{\text{Error}}} = \frac{SS_{\text{Error}}}{2^k(n - 1)}$

Under the null hypothesis $H_0: \text{Effect} = 0$ versus $H_1: \text{Effect} \neq 0$, the test statistic:

$F_0 = \frac{MS_{\text{Effect}}}{MS_{\text{Error}}}$

follows an $F$-distribution with $df_1 = 1$ and $df_2 = 2^k(n - 1)$. The null hypothesis is rejected at significance level $\alpha$ if:

$F_0 > F_{\alpha, 1, 2^k(n-1)} \quad \text{or if} \quad p\text{-value} < \alpha$

Unreplicated $2^k$ Designs and Pooling

When $n = 1$, $df_{\text{Error}} = 2^k(1 - 1) = 0$, meaning no internal estimate of pure error exists. In such cases, engineers employ two methods:

1. Daniel's Normal Probability Plot of Effects: Unimportant effects conform to a normal distribution centered at zero and fall along a straight line on a normal probability plot. Effects that diverge significantly from the line are identified as active.
2. Pooling (Sparsity-of-Effects Principle): High-order interactions (e.g., 3-factor and 4-factor terms) are assumed to be negligible and are pooled together to form an artificial error term with degrees of freedom equal to the sum of the pooled terms' degrees of freedom.

---

7. Fractional Factorial Designs ($2^{k-p}$)

As the number of factors $k$ increases, the run requirements of full factorial designs grow exponentially ($2^5 = 32$, $2^6 = 64$, $2^7 = 128$). According to the sparsity-of-effects principle (the Pareto rule of experimental design), systems are primarily driven by main effects and low-order (two-factor) interactions; three-factor and higher interactions are almost universally negligible.

A $2^{k-p}$ fractional factorial design runs a $\frac{1}{2^p}$ fraction of the full factorial, requiring $2^{k-p}$ runs while systematically confounding (aliasing) effects.

Design Generators and Defining Relations

To construct a fractional factorial:

1. Choose $k - p$ basic factors to form a full $2^{k-p}$ design matrix.
2. Confound the remaining $p$ factors with high-order interaction columns of the basic design. These assignment equations are called design generators.
3. Multiplying both sides of a generator by the generated factor yields a word in the defining relation (since $D \times D = D^2 = I$, where $I$ is the identity column of $+1$s).

Example: In a $2^{4-1}$ half-fraction ($2^{4-1} = 8$ runs), basic factors are $A, B, C$. We set $D = ABC$. Multiplying by $D$ gives the defining relation:

$I = ABCD$

Aliasing (Confounding) Structure

The alias of any factor is found by multiplying that factor across the defining relation using modulo 2 arithmetic ($X^2 = I$):

• For $A$: $A \cdot I = A \cdot ABCD \implies A = BCD$
• For $B$: $B \cdot I = B \cdot ABCD \implies B = ACD$
• For $AB$: $AB \cdot I = AB \cdot ABCD \implies AB = A^2B^2CD \implies AB = CD$

Thus, the calculated contrast for column $AB$ actually estimates the composite quantity $\beta_{AB} + \beta_{CD}$. It is impossible to determine whether a significant effect is caused by $AB$ or $CD$ without additional runs.

Design Resolution Levels

The resolution of a design is denoted by Roman numerals ($R = \text{III, IV, V}$) and equals the length of the shortest word in the defining relation:

• Resolution III Designs: No main effects are aliased with other main effects, but main effects are aliased with two-factor interactions (e.g., $2^{3-1}$ with $I = ABC \implies A = BC$). These are screening designs used to identify whether any factors are active, assuming interactions are zero.
• Resolution IV Designs: Main effects are unaliased with any two-factor interactions, but two-factor interactions are aliased with each other (e.g., $2^{4-1}$ with $I = ABCD \implies A = BCD$, but $AB = CD$). These designs cleanly isolate main effects.
• Resolution V Designs: No main effect or two-factor interaction is aliased with any other main effect or two-factor interaction; two-factor interactions are aliased only with three-factor interactions (e.g., $2^{5-1}$ with $I = ABCDE \implies A = BCDE, AB = CDE$). These provide high-fidelity modeling capability.

---

8. Complete Worked Numerical $2^3$ DOE Problem

Problem Formulation

An industrial manufacturing engineer wants to maximize the surface finish quality (measured as an index from 0 to 100, where higher is superior) of an aluminum aerospace component machined on a 5-axis CNC mill. The engineer investigates three controllable process parameters:

• Factor $A$ (Cutting Speed): Low = 1,500 rpm ($-1$), High = 2,500 rpm ($+1$)
• Factor $B$ (Feed Rate): Low = 150 mm/min ($-1$), High = 250 mm/min ($+1$)
• Factor $C$ (Depth of Cut): Low = 1.0 mm ($-1$), High = 2.5 mm ($+1$)

The experiment is executed with $n = 2$ true replicates in a completely randomized run order, yielding $N = 2^3 \times 2 = 16$ total runs.

Raw Experimental Data

Step 1: Calculate Contrasts

Applying the sign table columns to the treatment totals $y_{i\cdot}$:

• $\text{Contrast}_A = -46 + 62 - 56 + 88 - 38 + 54 - 48 + 78 = +94.0$
• $\text{Contrast}_B = -46 - 62 + 56 + 88 - 38 - 54 + 48 + 78 = +70.0$
• $\text{Contrast}_C = -46 - 62 - 56 - 88 + 38 + 54 + 48 + 78 = -34.0$
• $\text{Contrast}_{AB} = +46 - 62 - 56 + 88 + 38 - 54 - 48 + 78 = +30.0$
• $\text{Contrast}_{AC} = +46 - 62 + 56 - 88 - 38 + 54 - 48 + 78 = -2.0$
• $\text{Contrast}_{BC} = +46 + 62 - 56 - 88 - 38 - 54 + 48 + 78 = -2.0$
• $\text{Contrast}_{ABC} = -46 + 62 + 56 - 88 + 38 - 54 - 48 + 78 = -2.0$

Step 2: Calculate Effects and Sums of Squares

With $n = 2$ and $k = 3$, the effect divisor is $n \cdot 2^{k-1} = 2 \cdot 4 = 8$, and the sum of squares divisor is $n \cdot 2^k = 2 \cdot 8 = 16$:

• Factor $A$: $\text{Effect}_A = \frac{94.0}{8} = +11.75$ $SS_A = \frac{94.0^2}{16} = \frac{8836.0}{16} = 552.25$

• Factor $B$: $\text{Effect}_B = \frac{70.0}{8} = +8.75$ $SS_B = \frac{70.0^2}{16} = \frac{4900.0}{16} = 306.25$

• Factor $C$: $\text{Effect}_C = \frac{-34.0}{8} = -4.25$ $SS_C = \frac{(-34.0)^2}{16} = \frac{1156.0}{16} = 72.25$

• Interaction $AB$: $\text{Effect}_{AB} = \frac{30.0}{8} = +3.75$ $SS_{AB} = \frac{30.0^2}{16} = \frac{900.0}{16} = 56.25$

• Interaction $AC$: $\text{Effect}_{AC} = \frac{-2.0}{8} = -0.25$ $SS_{AC} = \frac{(-2.0)^2}{16} = \frac{4.0}{16} = 0.25$

• Interaction $BC$: $\text{Effect}_{BC} = \frac{-2.0}{8} = -0.25$ $SS_{BC} = \frac{(-2.0)^2}{16} = \frac{4.0}{16} = 0.25$

• Interaction $ABC$: $\text{Effect}_{ABC} = \frac{-2.0}{8} = -0.25$ $SS_{ABC} = \frac{(-2.0)^2}{16} = \frac{4.0}{16} = 0.25$

Step 3: Calculate Error Sum of Squares ($SS_{\text{Error}}$) and Total Sum of Squares ($SS_{\text{Total}}$)

For each treatment combination, the two replicates differ by exactly 2.0 units ($|y_{i1} - y_{i2}| = 2.0$). The sample variance within each treatment cell is:

$s_i^2 = \frac{(y_{i1} - \bar{y}_{i\cdot})^2 + (y_{i2} - \bar{y}_{i\cdot})^2}{2 - 1} = (-1.0)^2 + (1.0)^2 = 2.0$

Summing across all 8 treatment cells:

$SS_{\text{Error}} = \sum_{i=1}^8 (n - 1) s_i^2 = 8 \times (1 \times 2.0) = 16.00$

$df_{\text{Error}} = 2^3(n - 1) = 8 \times (2 - 1) = 8$

$MS_{\text{Error}} = \frac{16.00}{8} = 2.00$

Checking total variability:

$CF = \frac{Y_{\cdot\cdot}^2}{N} = \frac{470.0^2}{16} = \frac{220900.0}{16} = 13806.25$

$\sum_{i=1}^8 \sum_{j=1}^2 y_{ij}^2 = 22^2 + 24^2 + 32^2 + 30^2 + \dots + 38^2 + 40^2 = 14810.00$

$SS_{\text{Total}} = 14810.00 - 13806.25 = 1003.75$

Sum of all model terms:

$SS_{\text{Model}} = 552.25 + 306.25 + 72.25 + 56.25 + 0.25 + 0.25 + 0.25 = 987.75$

$SS_{\text{Total}} = SS_{\text{Model}} + SS_{\text{Error}} = 987.75 + 16.00 = 1003.75 \quad \text{(Identical Check)}$

Step 4: Complete ANOVA Table

Engineering Conclusions & Process Optimization

1. Active Factors: Cutting Speed ($A$), Feed Rate ($B$), and Depth of Cut ($C$) are all highly statistically significant main effects. Furthermore, the two-factor interaction $AB$ is strongly significant ($F_0 = 28.13 > 5.32$).
2. Interaction Synergy: The positive interaction coefficient $\beta_{AB} = +1.875$ indicates that increasing cutting speed produces an even greater improvement in surface quality when feed rate is also set at its high level.
3. Optimal Operating Setpoint: To maximize surface finish, the engineer sets Cutting Speed $A$ to high ($+1 = 2,500$ rpm), Feed Rate $B$ to high ($+1 = 250$ mm/min), and Depth of Cut $C$ to low ($-1 = 1.0$ mm, because $\text{Effect}_C = -4.25$ is negative, meaning lower depth increases quality). This corresponds to treatment combination $ab$, achieving an expected mean response of $\bar{y}_{ab} = 44.0$ index points.

---

9. Taguchi Robust Parameter Design

Genichi Taguchi's approach to process improvement aims to make a product or process robust, meaning insensitive to variation it cannot control. Its main ideas are:

• Quality loss function: loss grows with the square of the deviation from target, $L(y) = k(y - T)^2$, so reducing variation around the target has value even inside the specification.
• Control factors vs. noise factors: control factors are settings the engineer chooses, such as temperature or feed rate. Noise factors vary in production or use, such as ambient humidity or material lot, and are hard or costly to control.
• Orthogonal arrays: small, balanced fractional designs such as the $L_4$ (up to 3 two-level factors in 4 runs), $L_8$ (up to 7 two-level factors in 8 runs), $L_9$ (up to 4 three-level factors in 9 runs), and $L_{18}$ (mixed two- and three-level factors in 18 runs).
• Crossed arrays: an inner array of control-factor settings is crossed with an outer array of noise conditions, so each control setting is tested across the noise.

Signal-to-Noise (S/N) Ratios

Each inner-array row is summarized by an S/N ratio in decibels. The engineer chooses the control settings with the highest S/N ratio.

Example: A bond-strength test (larger-the-better) gives 42, 45, and 44 MPa across three noise conditions. The S/N ratio is $-10\log_{10}\left[\frac{1}{3}\left(\frac{1}{42^2} + \frac{1}{45^2} + \frac{1}{44^2}\right)\right] \approx 32.8$ dB. A setting with higher and more consistent strength would score higher.

Two-Step Optimization (Nominal-the-Best)

1. Choose control-factor levels that maximize the S/N ratio, which minimizes variation relative to the mean.
2. Use an adjustment factor, one that shifts the mean but barely affects the S/N ratio, to move the mean onto target.

Criticisms to know. Statisticians note that crossed arrays can need many runs, that S/N ratios can mix up location and dispersion effects, and that saturated orthogonal arrays confound main effects with interactions. A common alternative is a single combined design that models the response directly, including control-by-noise interactions. Exam questions usually test the vocabulary: control vs. noise factors, inner and outer arrays, S/N ratio types, and the two-step method.