10.3 Introduction to Design of Experiments (DOE) & Factorial Designs

10.3 Introduction to Design of Experiments (DOE) & Factorial Designs

Quick Summary: In the late Analyze and early Improve phases of DMAIC, continuous improvement teams transition from passive observational data collection to active experimentation. Design of Experiments (DOE) provides a structured, mathematically optimized methodology for simultaneously manipulating multiple input factors ($X$) to determine their individual and combined effects on a critical process response ($Y$). Traditional One-Factor-at-a-Time (OFAT) approaches are fundamentally flawed because they cannot detect interaction effects and require excessive trials. By deploying $2^k$ full factorial designs, utilizing the core experimental pillars—Randomization, Replication, and Blocking—and analyzing Main Effects and Interaction Plots, Green Belts can discover true root causes and dial in optimum operating recipes.

---

The Paradigm Shift: Passive Observation vs. Active Experimentation

Throughout the Measure and Analyze phases, Six Sigma teams primarily gather observational data—measuring the process as it naturally operates. While correlation and regression reveal associations in historical data, observational studies suffer from three severe scientific limitations:

1. Lurking Variables: Unmeasured factors drift undetected in the background (e.g., ambient humidity, raw material batch variations, operator shift changes, tool wear).
2. Multicollinearity: Inputs $X_1$ and $X_2$ may move together historically (e.g., furnace temperature and line pressure), making it mathematically impossible to separate their individual impacts on $Y$.
3. Absence of Causal Proof: Passive correlation demonstrates only that two variables co-vary; it can never prove that adjusting $X$ will cause $Y$ to change in a predictable direction.

               Passive Observation vs. Active Experimentation

   PASSIVE OBSERVATION (Measure / Analyze)     ACTIVE EXPERIMENTATION (Analyze / Improve)
  ┌─────────────────────────────────────┐    ┌─────────────────────────────────────┐
  │ • 'Listen' to the process as-is     │    │ • Intentionally intervene           │
  │ • Uncontrolled noise & lurking vars │    │ • Systematically set Xs to planned  │
  │ • Can only prove correlation        │    │   high (+1) and low (-1) levels     │
  │ • Retrospective / Historical data   │    │ • Proves direct CAUSALITY: Y = f(X) │
  └─────────────────────────────────────┘    └─────────────────────────────────────┘

Design of Experiments (DOE) represents active intervention: practitioners deliberately adjust process settings according to a pre-planned mathematical matrix, observe the resulting changes in output $Y$, and mathematically prove causality.

---

The Fatal Flaws of One-Factor-at-a-Time (OFAT)

Historically, engineers attempted experimentation using the One-Factor-at-a-Time (OFAT) method: holding all variables constant, varying Factor A to find its "best" setting, locking in Factor A, and then varying Factor B.

While conceptually intuitive, OFAT is an obsolete, invalid engineering practice due to three fatal flaws:

                      The Failure of OFAT Trajectory

     Factor B ▲
      (Temp)  │                         True Global Optimum (★)
              │                             ┌───┐
              │                             │ ★ │
              │                         ┌───┴───┴───┐
              │                 ┌───────┘  Yield    │
              │                 │          Contours │
              │            ┌────┘                   │
              │     OFAT   │                        │
              │     Locks  ▼                        │
              │     Here: (✖)                       │
              │            ▲                        │
              │            │ Step 1: Vary Temp only │
              │            │ (Holding Pressure Low) │
              └────────────┴────────────────────────┴────────► Factor A
              0                                                (Pressure)

1. Inability to Detect Interactions: OFAT operates on the naive assumption that the effect of Factor A is completely independent of Factor B. In reality, chemical, mechanical, and transactional processes are saturated with synergies and antagonisms. If temperature only boosts yield when pressure is high, OFAT will completely miss the effect if it tests temperature while pressure is held low.
2. Extreme Inefficiency: OFAT requires vastly more experimental runs to achieve the same statistical precision as a factorial design. In an OFAT experiment, data collected while varying Factor A provides zero information about Factor B.
3. Failure to Locate the Global Optimum: Because OFAT takes single-axis steps, it routinely gets trapped on local ridges, declaring a suboptimal setting as the "best possible" process condition (as shown at point ✖ above).

---

Fundamental Vocabulary of DOE

To communicate effectively and pass the CSSC Green Belt exam, candidates must master standardized DOE terminology:

---

Full Factorial Designs ($2^k$ Designs)

The foundational workhorse of industrial experimentation is the $2^k$ Full Factorial Design:

• $2$ represents the number of levels per factor (typically Low [$-1$] and High [$+1$]).
• $k$ represents the number of factors being evaluated.
• $N = 2^k$ represents the total number of unique treatment combinations (runs) required for a single replicate.

Common Factorial Sizes

• $2^2$ Design ($k = 2$ factors): $2^2 = \mathbf{4\text{ runs}}$ (evaluates Factors A, B, and interaction AB).
• $2^3$ Design ($k = 3$ factors): $2^3 = \mathbf{8\text{ runs}}$ (evaluates A, B, C, AB, AC, BC, ABC).
• $2^4$ Design ($k = 4$ factors): $2^4 = \mathbf{16\text{ runs}}$ (evaluates 4 main effects, 6 two-way interactions, 4 three-way interactions, and 1 four-way interaction).

Geometric Representation and Coded Design Matrix

A $2^2$ design is geometrically visualized as a square, while a $2^3$ design forms a cube, where each corner vertex represents a unique physical run:

                  Geometric Representation of 2² and 2³ Designs

         2² DESIGN (4 RUNS)                         2³ DESIGN (8 RUNS)
                                                      Factor C ▲
       Factor B ▲                                              │   (+1, +1, +1)
       (High) +1│  Run 3 (-1, +1)   Run 4 (+1, +1)             │      ┌─────────┐
                │     ○───────────────○                        │     ╱│        ╱│
                │     │               │                        │    ┌─────────┐ │
                │     │               │                        │    │ │       │ │
                │     │               │                        │    │ └───────┼─┘
        (Low) -1│     ○───────────────○                        │    │╱        │╱
                └─────┴───────────────┴────►                   │    └─────────┘
                     -1 (Low)        +1 (High)                 └────┴─────────┴─────► Factor A
                             Factor A                              ╱   Factor B

The Standard $2^3$ Coded Design Matrix

In a $2^3$ design, 8 treatment combinations evaluate three factors (A, B, C) and all four interaction terms:

Notice the vital property of orthogonality:

1. In every column, the number of $+1$s equals the number of $-1$s (balance).
2. The sum of the element-wise product of any two columns equals exactly zero: $\sum (c_{1i} \cdot c_{2i}) = 0$.
3. Orthogonality guarantees that every main effect and interaction effect is estimated completely independently of all other terms. There is zero multicollinearity.

---

The Three Foundational Pillars of DOE (Fisher's R-R-B)

To guarantee that experimental results are physically valid, unbiased, and statistically defensible, every designed experiment must adhere to the three classical principles formulated by Sir Ronald A. Fisher: Randomization, Replication, and Blocking (R-R-B).

                      The Three Pillars of DOE (R-R-B)

   1. RANDOMIZATION                2. REPLICATION                  3. BLOCKING
  ┌─────────────────────────┐     ┌─────────────────────────┐     ┌─────────────────────────┐
  │ Neutralizes Uncontrolled│     │ Quantifies Pure Error   │     │ Isolates Known Nuisance │
  │ Lurking Variables       │     │ & Setup Noise           │     │ Factors                 │
  │ (Run trials in random   │     │ (Re-run entire trials   │     │ ('Block what you can,   │
  │ order, not sorted)      │     │ from scratch)           │     │ randomize what you can't')
  └─────────────────────────┘     └─────────────────────────┘     └─────────────────────────┘

1. Randomization

• Principle: The execution sequence of experimental runs must be determined using a random number generator, never conducted in standard sorted order (e.g., never running all Low settings on Monday and all High settings on Tuesday).
• Purpose: Randomization averages out the effects of unmeasured, time-dependent lurking variables (e.g., machine warm-up, tool wear, ambient humidity fluctuations, operator fatigue). By randomizing run order, lurking variables transform from systematic bias into random statistical noise.

2. Replication

• Principle: Completely re-running the experimental treatment combinations multiple times, resetting machine parameters and loading fresh material for each replicate.
• Replication vs. Repetition (Exam Trap):
  • Replication: Tearing down the setup, re-dialing the machine settings, and processing a brand-new part. Captures total experimental error, including set-up variability and process noise.
  • Repetition (Repeated Measurement): Taking multiple consecutive sensor readings on the same physical part from a single run. Measures only measurement system error (Gage R&R) and severely underestimates process noise.
• Purpose: Provides an estimate of pure experimental error (the denominator $MS_{error}$ in ANOVA $F$-tests) and increases statistical power to detect subtle process differences.

3. Blocking

• Principle: Grouping homogeneous experimental units into distinct "blocks" to isolate known, unavoidable sources of variation that are not of primary scientific interest (e.g., raw material batches, different machine shifts, day-to-day weather).
• Operational Maxim: "Block what you can, and randomize what you cannot."
• Purpose: By accounting for block-to-block variation in the ANOVA model, that nuisance variability is removed from the experimental error term ($SS_{error}$), sharpening the signal-to-noise ratio and dramatically increasing the sensitivity of the experiment.

---

Center Points & Curvature Detection

A standard $2^k$ factorial tests factors at only two levels (Low $-1$ and High $+1$). Consequently, the mathematical model is forced to assume that the response between Low and High is strictly linear. If the true response is curved (quadratic), a 2-level design cannot detect it.

To detect curvature without incurring the massive expense of a 3-level design ($3^k$), practitioners add Center Points:

• Center Point Setting: Every continuous factor is set to its exact midpoint ($0$ coded unit): $x_{center} = (x_{low} + x_{high}) / 2$.
• Curvature Test: The mean of the center point runs ($\bar{Y}_C$) is compared against the mean of all factorial corner runs ($\bar{Y}_F$): $\text{Curvature Effect} = \bar{Y}_F - \bar{Y}_C$
• If $\bar{Y}_F$ and $\bar{Y}_C$ differ significantly ($p < 0.05$), statistically significant curvature exists in the response surface. This alerts the team that the linear model is inadequate and Response Surface Methodology (RSM, such as Central Composite Design) is required to map the peak optimum.

                       Detecting Curvature with Center Points

       LINEAR RESPONSE (No Curvature)             QUADRATIC CURVATURE DETECTED
    Y ▲                                        Y ▲             Center Points
      │                 Corner (+1)              │                 ○ ○ ○ (Y_C)
      │                 ○                        │               ╱       ╲
      │            ○ ○ (Center: Y_C)             │              ╱         ╲
      │          ╱                               │     Corner  ○           ○ Corner (+1)
      │         ╱                                │      (-1)   │           │
      │ Corner ○ (-1)                            │             │           │
      └────────┴────────┴─────────► Factor X     └─────────────┴───────────┴─────────► Factor X
              -1        0        +1                           -1           0          +1
           Y_C falls directly on line                   Y_C deviates significantly from line

---

Overview of Fractional Factorial Designs ($2^{k-p}$)

As the number of factors ($k$) grows, the runs required by a full factorial expand exponentially:

• $k = 5$ factors $\implies 2^5 = 32\text{ runs}$
• $k = 6$ factors $\implies 2^6 = 64\text{ runs}$
• $k = 7$ factors $\implies 2^7 = 128\text{ runs}$

In industrial settings where prototype parts cost thousands of dollars, running 64 or 128 trials is economically unfeasible.

The Sparsity of Effects Principle

Fractional factorials rely on the Sparsity of Effects Principle (the Pareto principle of experimental design):

In virtually all physical systems, process behavior is dominated by main effects and low-order two-factor interactions. Three-factor, four-factor, and higher-order interactions are almost always physically negligible or non-existent.

Fractional Factorial Mechanics ($2^{k-p}$)

A Fractional Factorial Design runs a carefully selected fraction ($1 / 2^p$) of the full factorial runs:

• $2^{k-1}$ (Half Fraction): Runs half the trials of a full factorial (e.g., $2^{4-1} = 8\text{ runs}$ instead of 16).
• $2^{k-2}$ (Quarter Fraction): Runs one-fourth the trials (e.g., $2^{5-2} = 8\text{ runs}$ instead of 32).

Confounding and Design Resolution

Because fewer runs are executed than full combinations, effects become confounded (aliased)—entangled together so their individual contributions cannot be separated mathematically.

Fractional designs are classified by Design Resolution (written in Roman numerals):

---

Step-by-Step Worked Planning Example: Designing a $2^3$ Factorial Experiment

Problem: A precision machining facility experiences surface roughness defects on aerospace titanium fittings. The Six Sigma Green Belt team designs an experiment to evaluate three continuous factors across 2 levels:

• Factor A (Spindle Speed): Low $= 1500\text{ RPM}$ ($-1$), High $= 3000\text{ RPM}$ ($+1$)
• Factor B (Feed Rate): Low $= 0.05\text{ mm/rev}$ ($-1$), High $= 0.15\text{ mm/rev}$ ($+1$)
• Factor C (Depth of Cut): Low $= 0.20\text{ mm}$ ($-1$), High $= 0.60\text{ mm}$ ($+1$)

The team decides to execute $n = 2$ complete replicates and add $n_c = 4$ center points to check for curvature and estimate pure experimental error. Calculate the total required runs, degrees of freedom breakdown, and specify the operational run protocol.

Step 1: Calculate Total Experimental Runs

• Factorial corner runs: $2^k \times n = 2^3 \times 2 = 8 \times 2 = 16\text{ runs}$
• Center point runs: $n_c = 4\text{ runs}$
• Total runs: $N_{total} = 16 + 4 = \mathbf{20\text{ runs}}$

Step 2: Determine Degrees of Freedom Allocation

Total degrees of freedom in the experiment: $DF_{total} = N_{total} - 1 = 20 - 1 = 19$.

Validation: $3 + 3 + 1 + 1 + 11 = 19$. The 11 degrees of freedom for pure error provide exceptional statistical power for ANOVA $F$-tests.

Step 3: Establish the Execution Protocol

1. Randomization: A random number generator assigns execution order from $1$ to $20$. Under no circumstances will runs be executed in standard order.
2. Replication Mechanics: For each run, the technician must completely disengage the tooling, wipe down the chuck, re-dial spindle speed and feed rate, and load a fresh titanium blank.
3. Center Point Placement: The 4 center point runs ($2250\text{ RPM}, 0.10\text{ mm/rev}, 0.40\text{ mm}$) are dispersed evenly throughout the randomized sequence to monitor process stability and detect baseline drift.

---

Critical Exam Traps to Avoid

• Trap 1: Confusing Replication with Repetition — Questions will describe measuring a single machined part 3 times with an optical comparator. This is repetition (measurement error), not replication. True replication requires resetting the machine and running fresh material from scratch.
• Trap 2: Selecting OFAT as the Best Experimental Strategy — When exam questions ask for the optimal way to optimize multi-factor processes with potential synergies, OFAT is always an incorrect distractor. Factorial DOE is the correct answer.
• Trap 3: Misidentifying Design Resolution Roman Numerals — Remember the rule of thumb: Resolution III means Main Effects + 2-Factor Interactions = $1 + 2 = 3$. Resolution IV means Main Effects + 3-Factor Interactions = $1 + 3 = 4$ (and $2 + 2 = 4$, so 2-way interactions are confounded with each other). Resolution V means $1 + 4 = 5$ and $2 + 3 = 5$.
• Trap 4: Omitting Center Points When Curvature is Suspected — A 2-level factorial cannot detect quadratic curvature on its own. Center points must be added to verify linearity without jumping directly to expensive 3-level designs.