Correlation & Linear Regression

Correlation & Linear Regression

Quick Answer: Correlation quantifies how strongly two continuous variables move together in a linear pattern (Pearson’s r from −1 to +1). Simple linear regression fits ŷ = b₀ + b₁x for estimation and prediction. A significant p-value supports a real linear association in the data; it does not by itself prove causation.

Correlation vs. Causation

In Analyze you often plot a candidate X against Y. A strong r is evidence of association, not proof that changing X will change Y.

Association can appear because X truly drives Y, Y drives X, a lurking Z drives both, the sample is coincidence, or both variables share a time trend. Green Belt rule: use correlation and regression to prioritize and quantify; confirm mechanism with process knowledge, DOE, or controlled interventions before locking Improve actions.

Scatter Plots First

Before computing r or fitting a line: plot Y vs X; look for linearity, outliers, clusters, and curvature. A curved cloud can yield r near zero despite a strong nonlinear link, or a moderate r with a poor linear fit. Scatter plots prevent misleading coefficients.

Pearson Correlation Coefficient

For paired observations ((x_i, y_i)), (i = 1, \ldots, n):

$$r = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum_{i=1}^{n}(x_i - \bar{x})^2 \, \sum_{i=1}^{n}(y_i - \bar{y})^2}}$$

Computational form:

$$r = \frac{n\sum x_i y_i - (\sum x_i)(\sum y_i)}{\sqrt{\bigl[n\sum x_i^2 - (\sum x_i)^2\bigr]\bigl[n\sum y_i^2 - (\sum y_i)^2\bigr]}}$$

Thresholds are guidelines. Pair r with the plot and with practical significance.

Properties: r is unitless and invariant to linear rescaling of X or Y; it measures linear association only; strength is about the pair, while regression slope Y on X is not the same as X on Y.

Simple Linear Regression

$$Y = \beta_0 + \beta_1 X + \varepsilon, \quad \hat{y} = b_0 + b_1 x$$ $$b_1 = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2} = r \, \frac{s_y}{s_x}, \quad b_0 = \bar{y} - b_1 \bar{x}$$

• b₁: estimated average change in Y per one-unit increase in X.
• b₀: fitted Y when x = 0 (often extrapolation if 0 is outside the operating range).

For one predictor, R² = r² — the fraction of sample Y variation explained by the linear fit. R² = 0.64 means 64% of observed Y variation in the sample is accounted for by the line, not that the project is 64% finished.

Estimation, Prediction, and Significance

• Estimation: for x in the observed range, (\hat{y} = b_0 + b_1 x) estimates the mean response at that x.
• Individual prediction: same point estimate is often used, but uncertainty for one future point is larger than for the mean.
• Extrapolation: x far outside the data range is risky.

For the slope (or testing population correlation ρ = 0 under standard assumptions):

• H₀: β₁ = 0 (no linear relationship)
• H₁: β₁ ≠ 0 (two-sided), or one-sided when the project cares about one direction

$$t = \frac{b_1}{SE(b_1)} \quad (\text{df} = n - 2)$$

The p-value is the probability under H₀ of a result at least as extreme as observed. At α = 0.05, p < 0.05 usually means the linear association is statistically significant. A tiny significant slope can still be practically useless; p ≥ 0.05 means insufficient evidence of a linear link—not proof that X never matters nonlinearly.

Worked Numerical Example

Oven temperature offset X (°C) vs bond strength Y (MPa), n = 5 (small for arithmetic; real projects use larger n):

(\bar{x} = 4), (\bar{y} = 13.2)

(\sum (x_i - \bar{x})^2 = 40)

(\sum (y_i - \bar{y})^2 = 22.80)

(\sum (x_i - \bar{x})(y_i - \bar{y}) = 30.0)

$$r = \frac{30.0}{\sqrt{40 \times 22.80}} = \frac{30.0}{\sqrt{912}} \approx 0.993$$ $$b_1 = \frac{30.0}{40} = 0.75, \quad b_0 = 13.2 - 0.75 \times 4 = 10.2$$ $$\hat{y} = 10.2 + 0.75x$$

Each 1 °C increase in offset is associated with about 0.75 MPa higher strength in the fitted range. At x = 5 (in range): (\hat{y} = 10.2 + 3.75 = 13.95) MPa. R² ≈ 0.987 in this nearly linear toy series—real process R² is usually lower. With r ≈ 0.993 and n = 5, the t-test of H₀: β₁ = 0 would give a small p-value under standard assumptions; the Green Belt still asks whether the link is causal, practical, and stable on the full line.

Reading p-Values Without Overclaiming

Always check residual plots when software is available: curved residuals mean the linear model is the wrong shape; fan-shaped residuals mean nonconstant variance.

Assumptions and DMAIC Fit

Least squares and usual p-values assume roughly linear mean structure, independent errors, roughly constant variance, and approximately normal errors for exact t procedures. Outliers can dominate b₁ and r. Multiple predictors need multiple regression (often deeper than core Green Belt practice); omitted variables still bias causal stories even when r looks clean.

1. Multi-vari and stratification locate major noise families.
2. Correlation/regression quantify continuous X–Y links.
3. Hypothesis tests and DOE confirm effects before Improve.

Expect CSSGB items that distinguish correlation from causation; interpret r, b₁, b₀, and R²; compute a simple slope or fitted value; and read a slope p-value without overclaiming process truth.