10.1 Linear Correlation & Pearson's Correlation Coefficient (r)

10.1 Linear Correlation & Pearson's Correlation Coefficient (r)

Quick Summary: In the Analyze phase of DMAIC, continuous improvement teams transition from qualitative brainstorming to quantitative validation of root causes ($Y = f(X)$). Correlation analysis measures the statistical association between two continuous variables. Pearson's correlation coefficient ($r$) quantifies the strength and direction of linear relationships on a standardized scale from $-1.0$ to $+1.0$. The coefficient of determination ($R^2 = r^2$) expresses the proportion of total variance in the output $Y$ explained by input $X$. However, correlation does not establish causation, and practitioners must guard against lethal analytical traps including non-linear dependencies, influential outliers, and synthetic artifacts illustrated by Anscombe's Quartet.

---

The Role of Correlation in the DMAIC Analyze Phase

During the Define and Measure phases, project teams identify the primary operational problem—the Critical to Quality (CTQ) response variable, designated as $Y$. In the Analyze phase, the objective shifts to isolating the vital few root causes—the input variables, designated as $X_1, X_2, \dots, X_k$—that dictate the behavior of $Y$.

Before investing capital in complex tooling modifications or process overhauls, Green Belts must prove mathematically that a suspected input $X$ shares a genuine association with output $Y$. Correlation analysis serves as the initial statistical filter in this journey.

                     The DMAIC Root Cause Funnel

   Brainstormed Inputs (Qualitative: Fishbone, 5 Whys)
   ┌─────────────────────────────────────────────────────────┐
   │ X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X12       │
   └───────────────────────────┬─────────────────────────────┘
                               ▼
   Correlation & Regression Screening (Quantitative Filtering)
   ┌─────────────────────────────────────────┐
   │ Evaluate Pearson's r, R², and p-values  │
   └─────────────────────┬───────────────────┘
                         ▼
   Vital Few Root Causes (Active DOE / Solution Design)
   ┌───────────────────────┐
   │ X2, X7 (Verified Xs)  │ ──▶ Transfers into IMPROVE Phase
   └───────────────────────┘

---

Scatter Plots: The Visual Prerequisite

A cardinal rule of Six Sigma data analytics states: Never calculate a numerical correlation coefficient without first examining a visual scatter plot of the raw data.

A scatter plot displays paired bivariate observations $(x_i, y_i)$ as Cartesian coordinates on a two-dimensional grid. Visual inspection immediately reveals patterns that numerical algorithms cannot distinguish:

                           Bivariate Scatter Patterns

     POSITIVE LINEAR              NEGATIVE LINEAR               ZERO CORRELATION
   Y ┌                  ▲       Y ┌ █                ▲        Y ┌   █   █   █        ▲
     │            █ █   │         │   █ █            │          │ █   █   █   █      │
     │        █ █       │         │       █ █        │          │   █   █   █   █    │
     │    █ █           │         │           █ █    │          │ █   █   █   █      │
     │  █               │         │               █  │          │   █   █   █        │
     └──────────────────┴─      └──────────────────┴─       └──────────────────┴─
     0                  X       0                  X        0                  X
     r ≈ +0.90 (Strong +)        r ≈ -0.90 (Strong -)         r ≈ 0.00 (No Linear)

     CURVILINEAR (NON-LINEAR)          ISOLATED OUTLIER             CLUSTERED GROUPS
   Y ┌         █ █      ▲       Y ┌                  ▲        Y ┌        ████        ▲
     │       █     █    │         │               █  │          │        ████        │
     │     █         █  │         │                  │          │                    │
     │   █             █│         │ █ █ █            │          │ ████               │
     │ █                │         │ █ █ █            │          │ ████               │
     └──────────────────┴─      └──────────────────┴─       └──────────────────┴─
     0                  X       0                  X        0                  X
     r ≈ 0.00 (Parabolic)        r inflated by 1 point        Simpson's Paradox Risk

1. Direction: Does $Y$ systematically increase as $X$ increases (positive slope), or does $Y$ decrease as $X$ increases (negative slope)?
2. Form: Is the data envelope straight (linear) or curved (exponential, logarithmic, parabolic)?
3. Strength: How tightly do individual data points cluster around an imaginary central trend line?
4. Unusual Observations: Are there isolated extreme points (outliers) that could distort statistical calculations?

---

Pearson's Product-Moment Correlation Coefficient ($r$)

Developed by Karl Pearson, the Pearson product-moment correlation coefficient—denoted as $\rho$ (rho) for a population and $r$ for a sample—quantifies the strength and direction of a linear relationship between two continuous variables.

Mathematical Formulation

Mathematically, $r$ represents the ratio of the sample covariance of $X$ and $Y$ to the product of their individual standard deviations:

$r = \frac{\sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum_{i=1}^n (x_i - \bar{x})^2} \sqrt{\sum_{i=1}^n (y_i - \bar{y})^2}} = \frac{SS_{xy}}{\sqrt{SS_{xx} \cdot SS_{yy}}}$

Where:

• $SS_{xy}$ is the sum of cross-products (covariation of $X$ and $Y$),
• $SS_{xx}$ is the sum of squared deviations for $X$,
• $SS_{yy}$ is the sum of squared deviations for $Y$.

Because both numerator and denominator are scaled by variability, $r$ is a dimensionless index. Changing the measurement units of $X$ or $Y$ (e.g., converting temperature from Celsius to Fahrenheit, or length from inches to millimeters) has zero effect on the value of $r$.

Theoretical Bounds and Scale

Pearson's $r$ is mathematically constrained to the closed interval:

$-1.0 \le r \le +1.0$

• $r = +1.0$ (Perfect Positive Linear Correlation): Every observation falls exactly on a straight line with a positive slope. As $X$ increases, $Y$ increases in exact lockstep.
• $r = -1.0$ (Perfect Negative Linear Correlation): Every observation falls exactly on a straight line with a negative slope. As $X$ increases, $Y$ decreases in exact lockstep.
• $r = 0.0$ (Zero Linear Correlation): There is zero linear association between $X$ and $Y$. Knowledge of $X$ provides no ability to predict $Y$ using a straight-line model.

Benchmarks for Interpreting Strength

While domain context dictates operational importance, the Council for Six Sigma Certification (CSSC) and industrial quality engineering apply standard benchmark classifications for correlation magnitude ($|r|$):

| Absolute Value Range ($|r|$) | Strength Classification | Operational Six Sigma Interpretation | | :--- | :--- | :--- | | $0.80 \le |r| \le 1.00$ | Strong (Very High) | Strong linear association; $X$ is a prime candidate for mathematical modeling and root cause control. | | $0.50 \le |r| < 0.80$ | Moderate | Substantial relationship exists, but considerable unexplained variation remains; secondary factors or noise are present. | | $0.20 \le |r| < 0.50$ | Weak | Slight linear tendency, but $X$ alone has minimal predictive utility. | | $0.00 \le |r| < 0.20$ | Very Weak to Negligible | No practical linear association; changes in $X$ do not explain meaningful variation in $Y$. |

Exam Tip: Correlation measures association, not slope! A steep line and a shallow line can both have $r = +1.00$, provided all points lie perfectly on their respective lines. The correlation coefficient $r$ tells you how tightly points cling to the line, not the steepness of the angle.

---

Spearman Rank Correlation ($r_s$ or $\rho_s$)

Pearson's $r$ requires two fundamental assumptions: both variables must be continuous, and their joint distribution must be approximately bivariate normal. When these assumptions fail, Green Belts deploy Spearman's Rank Correlation ($r_s$).

When to Use Spearman Rank Correlation

1. Ordinal Data: Data consists of ranks, ratings, or ordered categorical scales (e.g., customer satisfaction scores from 1 to 5, supplier audit tiers, defect severity ranks).
2. Severe Non-Normality: Continuous data displays extreme skewness, bi-modality, or heavy kurtosis that cannot be normalized.
3. Monotonic Non-Linear Relationships: The relationship between $X$ and $Y$ is consistently upward or consistently downward (monotonic), but follows an exponential or logarithmic curve rather than a straight line.

Operational Mechanics

Spearman's rank correlation replaces raw data values with their relative ranks (from $1$ to $n$). If ties occur, tied observations receive the average of the ranks they would have occupied. Pearson's formula is then applied directly to the ranked integers:

$r_s = 1 - \frac{6 \sum d_i^2}{n(n^2 - 1)}$

Where $d_i = \text{Rank}(x_i) - \text{Rank}(y_i)$ represents the difference between paired ranks for observation $i$.

---

The Coefficient of Determination ($R^2$)

While Pearson's $r$ indicates the strength of association, business leaders and project champions think in terms of explained variation. To quantify how much process variation is accounted for, Green Belts compute the Coefficient of Determination, denoted as $R^2$ (or $r^2$ in simple bivariate correlation):

$R^2 = (r)^2$

Formal Definition and Interpretation

The Coefficient of Determination ($R^2$) represents the proportion (or percentage) of the total variance in the dependent response variable ($Y$) that is mathematically explained or accounted for by its linear relationship with the independent predictor variable ($X$).

The value of $R^2$ ranges strictly from $0.0$ to $1.0$ (or $0%$ to $100%$):

• If $r = 0.80$, then $R^2 = (0.80)^2 = 0.64$ ($64%$). This proves that $64%$ of the total variation in $Y$ is explained by $X$, while the remaining $36%$ ($1 - R^2$) represents unexplained residual variation (process noise, unmeasured variables, or measurement error).
• If $r = -0.50$, then $R^2 = (-0.50)^2 = 0.25$ ($25%$). Notice that squaring removes the negative sign: $R^2$ is always non-negative.

               The Variance Pie: R² vs. Unexplained Noise

                 r = 0.80  ──▶  R² = 0.64 (64% Explained)

                        ┌────────────────────────┐
                        │████████████████████    │
                        │████████████████████    │
                        │████ EXPLAINED █████░░░░│ ░░ UNEXPLAINED
                        │████ BY INPUT X ████░░░░│    NOISE (1 - R²)
                        │████ (64%)      ████░░░░│    (36%)
                        │████████████████████    │
                        └────────────────────────┘

The Non-Linear Nature of $R^2$

A common executive mistake is assuming that $r = 0.80$ is twice as strong as $r = 0.40$. Examining $R^2$ reveals the truth:

• At $r = 0.40$, $R^2 = 0.16$ ($16%$ explained variation).
• At $r = 0.80$, $R^2 = 0.64$ ($64%$ explained variation).

An $r$ of $0.80$ actually explains four times ($64% / 16% = 4.0$) the variance of an $r$ of $0.40$. As $|r|$ approaches $1.0$, predictive power accelerates rapidly.

---

Hypothesis Testing for Linear Correlation

Observing a sample correlation of $r = 0.45$ in a small pilot dataset does not guarantee that the true population correlation ($\rho$) differs from zero. The observed association could be an artifact of random sampling noise.

To prove statistical significance, practitioners execute a two-tailed $t$-test:

1. Formulate the Hypotheses

• Null Hypothesis ($H_0$): $\rho = 0$ (There is no linear correlation in the population).
• Alternative Hypothesis ($H_a$): $\rho \ne 0$ (There is a statistically significant linear correlation).

2. Compute the Test Statistic ($t$)

The test statistic follows Student's $t$-distribution with $df = n - 2$ degrees of freedom:

$t = \frac{r \sqrt{n - 2}}{\sqrt{1 - r^2}}$

3. Apply the Decision Rule

• If the calculated $|t| > t_{\alpha/2, n-2}$ (or if the software $p$-value $\le \alpha$, typically $0.05$), Reject $H_0$ and conclude that a statistically significant linear relationship exists.
• If $p > \alpha$, Fail to Reject $H_0$; the evidence is insufficient to conclude that $\rho \ne 0$.

The Sample Size Leverage Effect

Sample size ($n$) heavily influences the $p$-value:

• With $n = 6$, an observed correlation of $r = 0.75$ yields $t = 0.75 \sqrt{4} / \sqrt{1 - 0.5625} = 1.50 / 0.6614 = 2.268$. For $df = 4$, the two-tailed critical value at $\alpha = 0.05$ is $2.776$. Because $2.268 < 2.776$ ($p = 0.086$), the team fails to reject $H_0$ despite a strong correlation!
• With $n = 500$, an observed correlation of $r = 0.12$ (practically useless in an engineering context) yields $t = 0.12 \sqrt{498} / \sqrt{1 - 0.0144} = 2.678 / 0.9928 = 2.70$ ($p = 0.007$). The test is statistically significant, yet $X$ explains only $R^2 = 1.4%$ of the variation in $Y$.

Key Principle: Never rely on statistical significance ($p$-value) alone. Always pair the $p$-value with the practical magnitude of $r$ and $R^2$.

---

Anscombe's Quartet: The Visual Imperative

In 1973, English statistician Francis Anscombe constructed four synthetic datasets to demonstrate both the necessity of graphing data before calculating summary statistics and the effect of outliers on statistical properties.

Each of the four datasets contains $n = 11$ paired observations $(X, Y)$. When analyzed using standard numerical formulas, all four datasets yield identical statistical outputs:

Despite having identical numerical summaries, inspecting their scatter plots reveals four radically different physical realities:

                      Anscombe's Quartet Visualized

   DATASET I: Ideal Linear              DATASET II: Non-Linear Parabolic
 Y ┌              █                   Y ┌             ███
   │          █     █                   │          ██     ██
   │      █ █                           │        ██         ██
   │    █                               │      ██             ██
   │  █                                 │    ██                 ██
   └───────────────────               └───────────────────
   0                  X               0                  X
   Standard linear relationship       Perfect curve; linear fit invalid

   DATASET III: Outlier Distorted       DATASET IV: High-Leverage Outlier
 Y ┌                                  Y ┌                   █ (Outlier)
   │             █ (Outlier)            │
   │         ████                       │    ████ (Vertical stack)
   │     ████                           │    ████
   │ ████                               │    ████
   └───────────────────               └───────────────────
   0                  X               0                  X
   True r = 1.0, ruined by 1 point    No relationship; 1 point drives fit

1. Dataset I: Fits a standard linear model with random, normally distributed error residuals around the regression line. Pearson's $r = 0.816$ accurately describes this relationship.
2. Dataset II: Exhibits a smooth, deterministic quadratic (parabolic) curve. The relationship between $X$ and $Y$ is perfect, but because it is non-linear, linear regression and Pearson's $r$ fail to capture its true nature.
3. Dataset III: Represents a perfect linear fit ($r = 1.0$) for 10 of the 11 points, but a single rogue outlier pulls the slope down and artificially reduces the correlation to $0.816$.
4. Dataset IV: Consists of 10 points stacked vertically at $x = 8.0$ with zero variation in $X$. A single high-leverage outlier located at $(x = 19, y = 12.5)$ single-handedly manufactures a correlation of $r = 0.816$ and creates a fictitious regression line where no relationship exists!

The Six Sigma Takeaway: A computer program will blindly process numbers and output $r = 0.816$ without alerting you that Dataset II is a curve or Dataset IV is an artificial artifact. Visual graphics are mandatory.

---

Critical Pitfalls in Correlation Analysis

Continuous improvement practitioners must avoid four major pitfalls when interpreting correlation:

1. The Non-Linear Trap

Pearson's $r$ evaluates linear association exclusively. If a biological or chemical process exhibits an inverted U-shape—where catalytic yield increases with temperature up to an optimum threshold and then crashes due to thermal decomposition—the calculated Pearson correlation across the entire operating window may equal $r = 0.00$. Inexperienced analysts will conclude that temperature does not affect yield. In reality, temperature exerts total control, but requires quadratic or polynomial modeling.

2. Outlier Distortion (Artificial or Masked Correlation)

As seen in Anscombe's Datasets III and IV, a single anomalous observation caused by a sensor spike or transcription error can:

• Transform an actual $r = 0.00$ into an apparent $r = 0.85$ (artificial correlation), or
• Transform an actual $r = 0.95$ into an apparent $r = 0.20$ (masked correlation).

Always apply robust outlier screening (such as studentized residuals or Cook's Distance) before finalizing correlation conclusions.

3. The Ecological Fallacy & Simpson's Paradox

The ecological fallacy occurs when inferences about individual units are deduced solely from aggregated group statistics. A related danger is Simpson's Paradox, where a correlation observed across aggregated data reverses direction when the data is disaggregated into operational subgroups (e.g., across machines, shifts, or suppliers):

                         Simpson's Paradox Visualized

      AGGREGATED DATA (Combined)               STRATIFIED BY SHIFT
   Y ┌                 ▲            Y ┌   Shift 1 (Neg)   Shift 2 (Neg)
     │           ███   │              │     █               █
     │        ███      │              │       █               █
     │     ███         │              │         █               █
     │  ███            │              │           █               █
     └─────────────────┴─           └────────────────────────────┴─
     0                 X            0                            X
     Apparent POSITIVE Trend        True NEGATIVE Trend in Both Shifts!

4. Correlation Does NOT Equal Causation

This is the cardinal rule of applied statistics. Observing a strong correlation ($|r| > 0.90$) between $X$ and $Y$ establishes mathematical co-movement, but never proves that $X$ causes $Y$. Three alternative mechanisms frequently generate high correlation without causation:

• Lurking / Confounding Variables ($Z$): Both $X$ and $Y$ are simultaneously driven by an unmeasured third factor. Example: Ice cream sales ($X$) and drowning incidents ($Y$) exhibit strong positive correlation ($r = 0.88$). Selling ice cream does not drown swimmers; outdoor summer temperature ($Z$) drives both variables independently.
• Reverse Causality: The analyst assumes $X$ drives $Y$, when in physical reality $Y$ drives $X$.
• Spurious Correlation: Pure coincidence in historical time-series data. Highly publicized examples include strong correlations between U.S. spending on science and suicides by hanging, or margarine consumption and divorce rates in Maine.

To prove causality, Green Belts must proceed beyond passive observational correlation to active manipulation through Design of Experiments (DOE).

---

Step-by-Step Worked Calculation Example

Problem: A Six Sigma Green Belt investigates whether preheating time ($X$, in minutes) correlates with final weld shear strength ($Y$, in MPa). A random sample of $n = 5$ production joints yields the following paired data:

Calculate the sample means, sums of squares, Pearson correlation coefficient $r$, coefficient of determination $R^2$, and conduct a hypothesis test at $\alpha = 0.05$.

Step 1: Compute Sample Means

$\bar{x} = \frac{2.0 + 4.0 + 5.0 + 7.0 + 8.0}{5} = \frac{26.0}{5} = 5.20\text{ min}$ $\bar{y} = \frac{35.0 + 45.0 + 50.0 + 65.0 + 70.0}{5} = \frac{265.0}{5} = 53.00\text{ MPa}$

Step 2: Set Up Deviations and Sums of Squares

Step 3: Compute Pearson's $r$

$r = \frac{SS_{xy}}{\sqrt{SS_{xx} \cdot SS_{yy}}} = \frac{137.00}{\sqrt{22.80 \times 830.00}} = \frac{137.00}{\sqrt{18924.0}} = \frac{137.00}{137.5645} \approx +0.9959$

Step 4: Compute Coefficient of Determination $R^2$

$R^2 = (0.9959)^2 \approx 0.9918 \quad (99.18\%)$
Interpretation: $99.18%$ of the total variation in weld shear strength is mathematically explained by preheating time.

Step 5: Test the Hypothesis ($H_0: \rho = 0$ vs $H_a: \rho \ne 0$)

$t = \frac{r \sqrt{n - 2}}{\sqrt{1 - r^2}} = \frac{0.9959 \sqrt{5 - 2}}{\sqrt{1 - 0.9918}} = \frac{0.9959 \times 1.73205}{\sqrt{0.0082}} = \frac{1.7249}{0.09055} \approx 19.05$

• Degrees of freedom: $df = n - 2 = 5 - 2 = 3$.
• From the Student's $t$-table, two-tailed critical value for $\alpha = 0.05$ with $df = 3$ is $t_{0.025, 3} = 3.182$.
• Since $|t| = 19.05 > 3.182$, we Reject $H_0$ ($p < 0.001$). There is a statistically significant, very strong positive linear correlation between preheating time and weld shear strength.

---

Critical Exam Traps to Avoid

• Trap 1: Confusing Correlation Strength ($r$) with Explained Variance ($R^2$) — Exam questions frequently ask: "If $r = 0.70$, what percentage of variation in $Y$ is explained by $X$?" Distractor choices will offer $70%$. The correct answer is $(0.70)^2 = 0.49 = 49%$.
• Trap 2: Assuming $r = 0$ Proves No Relationship Exists — A Pearson $r$ of zero indicates only that no linear relationship exists. The underlying data could form a perfect non-linear parabola or circle.
• Trap 3: Equating Steep Slope with High Correlation — A line with an equation $\hat{y} = 0.001x$ can have $r = 1.00$, while a steep line with $\hat{y} = 50x$ can have $r = 0.20$ if points are scattered widely around it.
• Trap 4: Claiming Statistically Significant Correlation Proves Causality — Rejection of $H_0: \rho = 0$ proves only that the observed linear pattern is unlikely due to random chance. It does not prove that $X$ causes $Y$.