6.7 Regression & Correlation Analysis

6.7 Regression & Correlation Analysis

In continuous process improvement, quality engineers frequently need to investigate relationships between process input variables ($x$, independent predictor) and product output parameters ($y$, dependent response). Correlation analysis quantifies the strength and direction of linear association, while regression analysis develops a mathematical model to predict $y$ from $x$ and optimize process settings.

---

1. Pearson Correlation Coefficient ($r$)

The sample Pearson correlation coefficient ($r$) measures the strength and direction of a linear relationship between two continuous variables:

$$r = \frac{S_{xy}}{\sqrt{S_{xx} S_{yy}}} = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum (x_i - \bar{x})^2 \sum (y_i - \bar{y})^2}}$$

Properties of $r$:

• Strictly bounded: $-1.0 \le r \le +1.0$.
• $r = +1.0$: Perfect positive linear association.
• $r = -1.0$: Perfect negative linear association.
• $r = 0.0$: No linear association (nonlinear patterns may still exist).

Testing Statistical Significance of $r$:

To test whether a sample correlation $r$ indicates a non-zero population correlation ($\rho \neq 0$), calculate the $t$-statistic with $df = n - 2$:

$$t_{calc} = \frac{r \sqrt{n - 2}}{\sqrt{1 - r^2}}$$

---

2. Coefficient of Determination ($R^2$)

The coefficient of determination ($R^2$) represents the proportion of total variability in response $y$ explained by the linear model:

$$R^2 = r^2 = \frac{SSR}{SST} = 1 - \frac{SSE}{SST}$$

• $SST = \sum (y_i - \bar{y})^2$: Total Sum of Squares
• $SSR = \sum (\hat{y}_i - \bar{y})^2$: Regression Sum of Squares (Explained variation)
• $SSE = \sum (y_i - \hat{y}_i)^2$: Error Sum of Squares (Unexplained variation)

Interpretation Example: If $r = 0.90$, then $R^2 = 0.81$. This means $81\%$ of the variation in the response variable is explained by the linear relationship with $x$, while $19\%$ is due to unexplained random noise or unmeasured process variables.

---

3. Simple Linear Regression Model & OLS Parameter Estimation

Simple linear regression models a linear relationship between predictor $x$ and response $y$:

$$y = \beta_0 + \beta_1 x + \varepsilon$$

where $\beta_0$ is the population y-intercept, $\beta_1$ is the population slope, and $\varepsilon \sim N(0, \sigma^2)$ is the random error term.

Ordinary Least Squares (OLS) Estimators

OLS estimates parameters $\hat{\beta}_0$ and $\hat{\beta}_1$ by minimizing the sum of squared residuals $\sum e_i^2 = \sum (y_i - \hat{y}_i)^2$:

$$\text{Estimated Slope:} \quad \hat{\beta}_1 = \frac{S_{xy}}{S_{xx}} = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2}$$ $$\text{Estimated Intercept:} \quad \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x}$$ $$\text{Fitted Regression Line:} \quad \hat{y} = \hat{\beta}_0 + \hat{\beta}_1 x$$

---

4. Model Assumptions & Residual Diagnostics (LINE)

For OLS regression estimates and hypothesis tests to be valid, four underlying assumptions regarding residuals ($e_i = y_i - \hat{y}_i$) must be satisfied (LINE acronym):

1. Linearity: The true relationship between $x$ and $y$ is linear.
2. Independence: Residuals are independent of one another (no autocorrelation over time).
3. Normality: Residuals are normally distributed with mean zero ($e_i \sim N(0, \sigma^2)$).
4. Equal Variance (Homoscedasticity): Residual variance is constant across all predicted values $\hat{y}$ and predictor values $x$.

Residual Plot Diagnostics:

• Residuals vs. Fitted Values ($\hat{y}$): Should show a random scatter of points around zero. A funnel pattern indicates non-constant variance (heteroscedasticity); a curved pattern indicates non-linearity.
• Normal Probability Plot of Residuals: Points should align closely along a straight diagonal line.

---

5. Worked Example: Regression of Adhesive Bond Strength on Cure Temperature

Scenario: A quality engineer evaluates how cure temperature ($x$ in $^\circ\text{C}$) affects adhesive bond strength ($y$ in $\text{MPa}$). A sample of $n = 5$ experimental runs produces the following summary statistics:

• $\bar{x} = 100^\circ\text{C}, \quad \bar{y} = 50\text{ MPa}$
• $S_{xx} = \sum (x_i - \bar{x})^2 = 250$
• $S_{yy} = \sum (y_i - \bar{y})^2 = 160$
• $S_{xy} = \sum (x_i - \bar{x})(y_i - \bar{y}) = 180$

Step-by-Step Regression Solution:

1. Calculate Slope ($\hat{\beta}_1$): $$\hat{\beta}_1 = \frac{S_{xy}}{S_{xx}} = \frac{180}{250} = 0.72\text{ MPa}/^\circ\text{C}$$
2. Calculate Intercept ($\hat{\beta}_0$): $$\hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x} = 50 - 0.72(100) = 50 - 72 = -22.0\text{ MPa}$$
3. Formulate Fitted Regression Equation: $$\hat{y} = -22.0 + 0.72 x$$
4. Calculate Pearson Correlation ($r$) and $R^2$: $$r = \frac{S_{xy}}{\sqrt{S_{xx} S_{yy}}} = \frac{180}{\sqrt{250 \times 160}} = \frac{180}{\sqrt{40000}} = \frac{180}{200} = 0.90$$ $$R^2 = (0.90)^2 = 0.81 \quad (81\% \text{ of variation explained})$$
5. Prediction Example: For a cure temperature of $x = 110^\circ\text{C}$: $$\hat{y} = -22.0 + 0.72(110) = -22.0 + 79.2 = 57.2\text{ MPa}$$