7.1 The Correlation Coefficient, Its Confidence Interval, and Causation

The Pearson correlation coefficient

$$r = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum (x_i - \bar{x})^2 \sum (y_i - \bar{y})^2}} = \frac{S_{xy}}{\sqrt{S_{xx} S_{yy}}}$$

$r$ measures the strength and direction of the linear association between two continuous variables and is bounded by $-1 \le r \le +1$. It is dimensionless and unaffected by linear rescaling, so converting millimetres to inches changes nothing.

These bands are conventions, not thresholds. In a designed experiment on a machined dimension, $r = 0.85$ may be disappointing; in a study of human behaviour, $r = 0.40$ may be a major finding.

The coefficient of determination

$$r^2 = \text{proportion of the variation in } Y \text{ explained by the linear relationship with } X$$

This is the interpretation that matters practically. $r = 0.70$ sounds impressive until you square it: $r^2 = 0.49$, so 51% of the variation remains unexplained. Reporting $r$ alone systematically overstates how much of the problem a factor accounts for, which is why the Analyze phase should prioritize factors by $r^2$, not by $r$.

Worked example

Six paired observations of oven temperature ($X$, degrees C) and tensile strength ($Y$, MPa):

$\bar{x} = 192.5$, $\bar{y} = 432.5$. Computing the sums of squares gives $S_{xx} = 437.5$, $S_{yy} = 1229.5$, and $S_{xy} = 727.5$. Then:

$$r = \frac{727.5}{\sqrt{437.5 \times 1229.5}} = \frac{727.5}{733.42} = 0.9919$$

$r^2 = 0.9839$, so 98.4% of the variation in tensile strength over this temperature range is explained by the linear relationship. Note the qualifier over this range -- nothing here supports extrapolation to 250 C.

Testing and estimating

Testing whether rho differs from zero

$$t = \frac{r\sqrt{n-2}}{\sqrt{1-r^2}}, \qquad df = n - 2$$

Reject $H_0: \rho = 0$ when $|t|$ exceeds the critical value. For the example above, $t = 0.9919\sqrt{4}/\sqrt{1-0.9839} = 1.9839/0.1268 = 15.6$ with 4 degrees of freedom, which is overwhelmingly significant.

Confidence interval via Fisher's z transformation

The sampling distribution of $r$ is skewed whenever $\rho \ne 0$, because $r$ is bounded at $\pm 1$. Fisher's transformation removes the skew:

$$z' = \frac{1}{2}\ln\left(\frac{1+r}{1-r}\right), \qquad SE_{z'} = \frac{1}{\sqrt{n-3}}$$

Build the interval on the $z'$ scale, $z' \pm 1.96 \, SE_{z'}$ for 95% confidence, then transform each endpoint back:

$$r = \frac{e^{2z'} - 1}{e^{2z'} + 1}$$

Two practical points. The interval is asymmetric around $r$ on the original scale, and it requires $n > 3$. With small samples it is remarkably wide -- with $n = 10$ and $r = 0.60$, the 95% interval runs roughly from -0.04 to 0.89, which is not evidence of a strong relationship.

Assumptions and what breaks them

Always plot the data before reading $r$. Anscombe's quartet -- four data sets with identical correlation, means, and regression lines but radically different shapes -- exists precisely to make this point. A scatter plot takes seconds and prevents the two most common failures: a curved relationship reported as no relationship, and a single outlier reported as a strong relationship.

Restricted range deserves special attention in Six Sigma work, because it is built into the way we collect data. If a process is tightly controlled between 199 and 201 C, a study of temperature versus strength within that band will show little correlation even if temperature is a dominant factor across a wider range. The absence of correlation in observational process data is weak evidence that a factor does not matter; a designed experiment that deliberately widens the range is the correct test.

Correlation and causation

An observed correlation is consistent with several explanations, only one of which is direct causation:

1. X causes Y -- the conclusion usually wanted.
2. Y causes X -- reverse causation.
3. A third variable Z causes both -- confounding.
4. Selection effect -- the sampling procedure created the association.
5. Coincidence -- with enough variables examined, spurious correlations appear.

The last is worth quantifying: testing 20 unrelated factor pairs at $\alpha = 0.05$ produces on average one significant result by chance alone. Data-mining a wide process data set for correlations without adjusting for multiplicity reliably generates false leads.

Establishing causation

Randomization in a designed experiment is what neutralizes unknown confounders, which is why the Improve phase of DMAIC is built around DOE rather than around further observational analysis. A correlation found in the Analyze phase is a hypothesis to be tested, not a root cause to be acted on.