2.2 Hypothesis Testing, ANOVA, and Statistical Quality Control

Hypothesis Testing and Decision Errors

Chemical engineers must make decisions based on experimental data that are subject to uncertainty. Statistical hypothesis testing provides a mathematical framework required to distinguish random noise from true physical effects, such as catalyst enhancements, reactor performance variations, or product specification drifts.

A hypothesis test is a formal procedure to decide between two competing claims about a population parameter:

• Null Hypothesis ($H_0$): The status quo or default assumption that there is no difference, change, or effect (e.g., a new catalyst yields the same mean conversion as the old catalyst, $H_0: \mu = \mu_0$).
• Alternative Hypothesis ($H_1$ or $H_a$): The claim the engineer wants to prove (e.g., the new catalyst increases mean conversion, $H_1: \mu > \mu_0$, which is a one-tailed test; or the conversion is simply different, $H_1: \mu \neq \mu_0$, which is a two-tailed test).

When conducting hypothesis tests, two types of decision errors can occur:

• Type I Error ($\alpha$): Rejecting the null hypothesis when it is actually true (false positive). The probability of a Type I error is the significance level $\alpha$, typically set at $0.05$ or $0.01$.
• Type II Error ($\beta$): Failing to reject the null hypothesis when it is actually false (false negative). The probability of a Type II error is denoted by $\beta$.
• Power of the Test ($1 - \beta$): The probability of correctly rejecting the null hypothesis when it is false. Increasing the sample size $n$ generally increases the power of the test.

Student's t-Tests for Means Comparison

To compare sample means when the population standard deviation is unknown, Student's t-tests are applied. The choice of the test depends on the experimental design:

• One-Sample t-Test: Compares the sample mean $\bar{X}$ to a known reference value $\mu_0$: $t_{\text{calc}} = \frac{\bar{X} - \mu_0}{s / \sqrt{n}}$ where $s$ is the sample standard deviation and $n$ is the sample size. The calculated value is compared to critical values from the t-distribution table with $\nu = n-1$ degrees of freedom. If $|t_{\text{calc}}| > t_{\alpha/2, n-1}$ (for a two-tailed test), the difference is statistically significant.
• Two-Sample Independent t-Test (Equal Variances): Compares the means of two independent groups ($\bar{X}_1$ and $\bar{X}_2$). The pooled standard deviation $s_p$ is calculated as: $s_p = \sqrt{\frac{(n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + n_2 - 2}}$ The test statistic is: $t_{\text{calc}} = \frac{\bar{X}_1 - \bar{X}_2}{s_p \sqrt{\frac{1}{n_1} + \frac{1}{n_2}}}$ with degrees of freedom $\nu = n_1 + n_2 - 2$.
• Paired t-Test: Used when comparing dependent observations (e.g., measuring reactor concentration before and after a catalyst change on the same system). The differences $d_i = X_{1,i} - X_{2,i}$ are calculated, and the test statistic is $t_{\text{calc}} = \frac{\bar{d} - \mu_d}{s_d / \sqrt{n}}$ with $\nu = n-1$ degrees of freedom, where $\bar{d}$ is the mean of the differences and $s_d$ is their standard deviation.

Analysis of Variance (ANOVA)

Analysis of Variance (ANOVA) generalizes the comparison of means to three or more treatment groups (e.g., comparing reactor yields under three different temperature regimes). One-way ANOVA tests the null hypothesis that all group means are equal: $H_0: \mu_1 = \mu_2 = \dots = \mu_k$ against the alternative $H_1$: at least one group mean is different.

ANOVA partitions the total variation in the data set into two components:

1. Sum of Squares for Treatments ($SS_{\text{Treatments}}$ or $SS_{\text{Between}}$): Measures variation between the group means and the grand mean.
2. Sum of Squares for Error ($SS_{\text{Error}}$ or $SS_{\text{Within}}$): Measures variation within each group due to random noise.

The total sum of squares is $SS_{\text{Total}} = SS_{\text{Treatments}} + SS_{\text{Error}}$. Mean squares ($MS$) are calculated by dividing the sum of squares by their respective degrees of freedom ($df$): $MS_{\text{Treatments}} = \frac{SS_{\text{Treatments}}}{k-1}$ and $MS_{\text{Error}} = \frac{SS_{\text{Error}}}{n-k}$, where $k$ is the number of groups and $n$ is the total number of observations. The test statistic is the F-ratio: $F_{\text{calc}} = \frac{MS_{\text{Treatments}}}{MS_{\text{Error}}}$ If $F_{\text{calc}}$ is greater than the critical value $F_{\alpha, k-1, n-k}$ from the F-distribution table in the FE Reference Handbook, the null hypothesis is rejected, indicating a statistically significant difference between the means.

Expected Value in Decision Analysis

Chemical engineers often face economic decisions under uncertainty, such as selecting a process design route, purchasing redundant equipment, or sizing emergency backup units. The expected value $E(X)$ of a discrete random variable is the probability-weighted average of all possible outcomes: $E(X) = \sum_{i=1}^{m} x_i P(x_i)$ where $x_i$ represents the value of outcome $i$ and $P(x_i)$ is the probability of that outcome.

For example, when evaluating whether to install a backup cooling water pump, the engineer compares the guaranteed cost of installation to the expected cost of production downtime caused by main pump failure: $\text{Expected Cost of Failure} = P(\text{Failure}) \times \text{Cost of Halt}$ If the backup pump costs $15,000 to install, and the expected cost of not installing it is $0.12 \times 100,000 = 12,000$ USD, then on a purely expected value basis, the installation cost exceeds the expected failure cost, and the plant should not install the backup pump. However, engineers must often balance expected value calculations with safety margins and risk-averse strategies.

Outlier Testing (Q-Test and Grubbs' Test)

Outliers are extreme observations that deviate significantly from the rest of the data. They can arise from experimental error, equipment malfunction, or transcription mistakes. Engineers must test whether to statistically discard these points rather than let them bias calculations.

• Dixon's Q-Test: Used for small data sets ($3 \le n \le 30$) to test a single suspect outlier. The data are arranged in ascending order, and the Q-ratio is calculated as: $Q = \frac{\text{gap}}{\text{range}} = \frac{|x_{\text{suspect}} - x_{\text{nearest}}|}{x_{\text{max}} - x_{\text{min}}}$ If $Q_{\text{calc}} > Q_{\text{critical}}$ (from the Q-table at a given confidence level, typically 95%), the outlier is discarded.
• Grubbs' Test: Recommended for larger samples to detect a single outlier. The Grubbs test statistic $G$ is: $G = \frac{|x_{\text{suspect}} - \bar{X}|}{s}$ If $G_{\text{calc}} > G_{\text{critical}}$ from Grubbs' table, the suspect data point is rejected. Once discarded, the mean and standard deviation must be recalculated using the remaining $n-1$ data points.

Statistical Quality Control and Control Charts

Statistical Quality Control (SQC) uses control charts to monitor a process over time to detect whether variation is due to common causes (random noise) or assignable causes (process drifts, feed changes, or equipment wear). For continuous variables (such as product viscosity or moisture content), engineers use Shewhart Control Charts, specifically the $\bar{X}$ (mean) chart to monitor process centering and the $R$ (range) chart to monitor process dispersion.

• Subgroup Sampling: Samples of size $n$ are taken at regular intervals. For each subgroup, the mean $\bar{X}_j$ and range $R_j$ are calculated.
• Center Lines (CL): $\text{Grand Mean } \bar{\bar{X}} = \frac{1}{m} \sum_{j=1}^{m} \bar{X}_j \quad \text{and} \quad \text{Average Range } \bar{R} = \frac{1}{m} \sum_{j=1}^{m} R_j$
• Control Limits: 3-sigma control limits are established using standard control chart factor tables from the FE Reference Handbook:
  • $\bar{X}$ Chart Limits: $\text{UCL}{\bar{X}} = \bar{\bar{X}} + A_2 \bar{R}$ and $\text{LCL}{\bar{X}} = \bar{\bar{X}} - A_2 \bar{R}$
  • $R$ Chart Limits: $\text{UCL}_R = D_4 \bar{R}$ and $\text{LCL}_R = D_3 \bar{R}$ where $A_2$, $D_3$, and $D_4$ are constants dependent on the subgroup sample size $n$. If any subgroup mean or range falls outside these limits, the process is considered "out of control," prompting investigation.