7.2 Bivariate Data, Scatterplots, Correlation & Linear Regression

7.2 Bivariate Data, Scatterplots, Correlation & Linear Regression

Investigating relationships between two quantitative variables is a cornerstone of secondary mathematics education. Under the NBPTS standards, accomplished teachers guide students from informal visual impressions of scatterplots to rigorous algebraic and statistical modeling. This involves understanding the mathematical mechanisms governing the Pearson correlation coefficient $r$, deriving the least-squares regression line (LSRL), evaluating model assumptions via residual plots, and guarding against the pervasive traps of confounding, extrapolation, and misinterpreting influential observations.

---

1. Scatterplots & Characterizing Bivariate Associations

A scatterplot displays bivariate quantitative data $(x_i, y_i)$ on a Cartesian coordinate plane. Convention assigns:

• Explanatory Variable ($x$, Independent/Predictor): Plotted along the horizontal axis; hypothesized to explain, predict, or influence changes in the response.
• Response Variable ($y$, Dependent/Criterion): Plotted along the vertical axis; measures the outcome of interest.

Four Essential Descriptors of Bivariate Data

When describing a scatterplot, secondary students must systematically address four key characteristics:

1. Direction: A positive association indicates that above-average values of $x$ tend to accompany above-average values of $y$; a negative association indicates that above-average values of $x$ accompany below-average values of $y$.
2. Form: Linear, curvilinear (quadratic, exponential, logarithmic), or clustered into distinct subgroups.
3. Strength: The degree of adherence of the points to the underlying form (strong, moderate, or weak).
4. Unusual Features: Outliers, influential points, high-leverage points, or pronounced gaps.

---

2. Pearson Product-Moment Correlation Coefficient ($r$)

The Pearson correlation coefficient $r$ provides an objective, standardized metric of the strength and direction of a linear relationship between two quantitative variables.

Mathematical Formulation

For $n$ paired observations $(x_1, y_1), (x_2, y_2), \dots, (x_n, y_n)$ with sample standard deviations $s_x$ and $s_y$: $r = \frac{1}{n - 1} \sum_{i=1}^{n} \left( \frac{x_i - \bar{x}}{s_x} \right) \left( \frac{y_i - \bar{y}}{s_y} \right) = \frac{\sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum_{i=1}^{n} (x_i - \bar{x})^2 \sum_{i=1}^{n} (y_i - \bar{y})^2}}$

Notice that $r$ is the average product of standardized $z$-scores: $r = \frac{1}{n-1} \sum z_{x_i} z_{y_i}$. When a data point falls in Quadrant I or III relative to the centroid $(\bar{x}, \bar{y})$, both $z$-scores share the same algebraic sign, contributing a positive product to the sum. In Quadrants II and IV, the $z$-scores have opposite signs, contributing a negative product.

Critical Mathematical Properties of $r$

• Bounded Range: $-1 \le r \le 1$. Values of $r = 1$ and $r = -1$ represent perfect positive and perfect negative collinearity, respectively. A value of $r = 0$ denotes no linear association.
• Dimensionless (Scale Invariance): Because $r$ is computed from standardized $z$-scores, it has no units of measurement. Transforming units (e.g., converting centimeters to inches or Celsius to Fahrenheit) leaves $r$ completely unchanged, provided the scale factor is positive ($a > 0, c > 0$). If one variable is multiplied by a negative constant, $r$ reverses sign.
• Symmetry: $r_{xy} = r_{yx}$. Switching the roles of explanatory and response variables does not alter the correlation coefficient.
• Linearity Restriction: $r$ measures strictly linear association. A deterministic non-linear relationship (e.g., $y = x^2$ on the symmetric domain $[-3, 3]$) yields $r = 0$ despite perfect functional predictability.
• Non-Resistance: Like the sample mean, $r$ is non-resistant. A single anomalous observation far from the bivariate centroid can drastically inflate or deflate $r$.

---

3. The Least-Squares Regression Line (LSRL)

When a scatterplot demonstrates a reasonably linear trend, linear regression models the response $y$ as a linear function of $x$: $\hat{y} = b_0 + b_1 x$ where $\hat{y}$ (y-hat) represents the predicted value of the response variable for a given value of $x$, $b_1$ is the slope, and $b_0$ is the $y$-intercept.

The Least-Squares Criterion

For each observation $(x_i, y_i)$, the vertical error or residual is defined as: $e_i = y_i - \hat{y}_i = \text{observed } y - \text{predicted } y$

The least-squares regression line is defined as the unique line that minimizes the sum of squared vertical residuals (SSE): $\min_{b_0, b_1} \sum_{i=1}^{n} e_i^2 = \min_{b_0, b_1} \sum_{i=1}^{n} \left[ y_i - (b_0 + b_1 x_i) \right]^2$

Formulas for Slope ($b_1$) and Intercept ($b_0$)

Setting the partial derivatives $\frac{\partial \text{SSE}}{\partial b_0} = 0$ and $\frac{\partial \text{SSE}}{\partial b_1} = 0$ yields the normal equations. Solving these equations produces the fundamental parameter formulas: $b_1 = r \left( \frac{s_y}{s_x} \right)$ $b_0 = \bar{y} - b_1 \bar{x}$

[!IMPORTANT] The Centroid Property: Rearranging the intercept formula gives $\bar{y} = b_0 + b_1 \bar{x}$. This proves mathematically that the least-squares regression line always passes through the centroid point $(\bar{x}, \bar{y})$ of the dataset.

Contextual Interpretation of Regression Parameters

• Interpreting the Slope ($b_1$): "For each additional 1-unit increase in the explanatory variable $x$, the model predicts an average change of $b_1$ units in the response variable $y$."
• Interpreting the $y$-Intercept ($b_0$): "When the explanatory variable $x$ is 0 units, the model predicts the response variable $y$ will be $b_0$ units." (Teachers must prompt students to assess whether $x = 0$ is physically meaningful and lies within the observed domain).

---

4. The Coefficient of Determination ($r^2$)

The coefficient of determination, denoted $r^2$, evaluates the explanatory adequacy of the regression model. In simple linear regression with an intercept, $r^2$ is precisely the square of Pearson's correlation coefficient.

Partitioning the Sum of Squares

The total variation in the response variable around its mean is partitioned into two additive components: $\text{SST} = \text{SSM} + \text{SSE}$ $\sum_{i=1}^{n} (y_i - \bar{y})^2 = \sum_{i=1}^{n} (\hat{y}_i - \bar{y})^2 + \sum_{i=1}^{n} (y_i - \hat{y}_i)^2$ where:

• $\text{SST}$ (Total Sum of Squares): Total variation in $y$ without using $x$.
• $\text{SSM}$ (Model Sum of Squares): Variation explained by the linear regression line.
• $\text{SSE}$ (Error/Residual Sum of Squares): Unexplained residual scatter.

$r^2 = \frac{\text{SSM}}{\text{SST}} = 1 - \frac{\text{SSE}}{\text{SST}}$

Standard Pedagogical Interpretation Template

Accomplished teachers train students to use the precise interpretative phrasing:

"$r^2 \times 100%$ of the total variation in the response variable ($y$) is explained by the linear relationship with the explanatory variable ($x$)."

---

5. Residual Analysis & Diagnostic Plots

Calculating a high $r$ or $r^2$ is insufficient to claim that a linear model is appropriate. A linear model can produce $r = 0.95$ on data exhibiting severe quadratic curvature. The primary diagnostic tool for model validation is the residual plot.

Fundamental Properties of Residuals

1. $\sum_{i=1}^{n} e_i = 0$ (the sum of residuals from an LSRL with an intercept is always exactly zero).
2. $\bar{e} = 0$ (the mean residual is zero).

Diagnosing Residual Patterns

A residual plot graphs the explanatory variable $x_i$ (or predicted values $\hat{y}_i$) on the horizontal axis against the residuals $e_i$ on the vertical axis:

+-----------------------------------------------------------------------------------+
|                         RESIDUAL PLOT DIAGNOSTIC MATRIX                           |
+-----------------------+-----------------------------+-----------------------------+
| Observed Pattern      | Statistical Diagnosis       | Remedial Modeling Action    |
+-----------------------+-----------------------------+-----------------------------+
| Uniform random        | Linear model is appropriate | Retain linear model;        |
| scatter around e = 0  | (Homoscedasticity confirmed)| proceed with inference      |
+-----------------------+-----------------------------+-----------------------------+
| Curved pattern        | Non-linear relationship     | Reject linear model; apply  |
| (U-shape / Inverted U)| (Systematic model inadequacy| re-expression (log, power)  |
+-----------------------+-----------------------------+-----------------------------+
| Fan / Funnel shape    | Heteroscedasticity          | Logarithmic transformation  |
| (Spread widens/narrows| (Non-constant error variance| on y to stabilize variance  |
+-----------------------+-----------------------------+-----------------------------+

---

6. Outliers, Leverage, and Influential Points

In bivariate regression, unusual points exhibit distinct geometric properties and consequences:

Bivariate Outlier

An observation with a large vertical residual $|e_i|$. It falls far from the general linear trend formed by the majority of the data points.

High-Leverage Point

An observation whose explanatory value $x_i$ lies substantially far from the centroid mean $\bar{x}$.

• Leverage Property: Leverage depends solely on the $x$-value, completely independent of the $y$-value.
• A high-leverage point acts like a heavy weight at the end of a long lever arm. If its $y$-value aligns with the linear trend established by the rest of the data, it has a small residual, does not alter the slope, but artificially inflates $r$ and $r^2$.

Influential Point

An observation whose inclusion or omission substantially alters the regression parameters (slope $b_1$, intercept $b_0$, or correlation $r$). A point is typically influential if it combines high leverage with an inconsistent $y$-value that fails to follow the trend of the remaining data.

---

7. Secondary Pedagogical Traps & Instructional Interventions

Trap 1: Correlation Does Not Imply Causation

Students routinely conclude that a strong positive correlation between two variables proves that changes in $x$ cause changes in $y$.

• Pedagogical Intervention: Present real-world examples driven by lurking / confounding variables (e.g., the strong positive correlation between ice cream sales and drowning incidents, both driven by the confounding variable of summer outdoor temperatures). Emphasize that only a well-designed, randomized controlled experiment can establish causal inference.

Trap 2: The Hazard of Extrapolation

Students frequently use regression equations to make predictions for $x$-values far outside the range of observed data.

• Pedagogical Intervention: Illustrate the growth rate of a human infant. A linear regression predicting child height from age between 1 and 4 years will predict an adult height of 14 feet at age 35. Explain that relationships verified to be linear within a specific experimental domain may become non-linear, saturate, or reverse entirely beyond the observed data range.

Trap 3: Switching $x$ and $y$ in Regression

Students often assume that the regression line of $y$ on $x$ can simply be solved algebraically for $x$ to yield the regression of $x$ on $y$.

• Pedagogical Intervention: Clarify that least-squares regression minimizes vertical distances ($y - \hat{y}$), whereas regressing $x$ on $y$ minimizes horizontal distances ($x - \hat{x}$). Because $b_{1, y \text{ on } x} = r\left(\frac{s_y}{s_x}\right)$ while $b_{1, x \text{ on } y} = r\left(\frac{s_x}{s_y}\right)$, the lines are geometrically distinct whenever $|r| < 1$.