9.1 Correlation & Regression Analysis

Correlation & Regression Analysis in Healthcare

Advanced statistical modeling forms the core methodological foundation for clinical effectiveness research, healthcare operational forecasting, quality benchmarking, and predictive risk stratification. For a Certified Health Data Analyst (CHDA), raw descriptive summaries and simple bivariate hypothesis tests are rarely sufficient to capture the multifaceted, highly confounded realities of modern healthcare delivery. Inpatient length of stay, total episode costs, chronic disease complications, and 30-day hospital readmissions are governed by complex networks of interacting demographic, clinical, pharmacological, and socioeconomic factors.

Mastering correlation analysis, Simple Linear Regression, Multiple Linear Regression, diagnostic residual analysis, and Logistic Regression equips the analyst to isolate true clinical signals from confounding noise, quantify predictive certainty, and build mathematically sound models that withstand regulatory and clinical scrutiny.

---

1. Correlation Analysis: Association vs. Causation

Correlation quantifies the degree to which two variables fluctuate together in a synchronized manner. However, a foundational tenet of health data analytics is that correlation does not imply causation. An empirical correlation between two clinical or operational variables may arise from direct causation, reverse causation, bidirectional feedback, or mutual association with an unobserved confounding factor (omitted variable bias or spurious correlation).

+---------------------------------------------------------------------------------------------------+
|                                 CORRELATION VS. CAUSATION PARADIGM                                |
+---------------------------------------------------------------------------------------------------+
| 1. DIRECT CAUSALITY:       [Elevated Serum HbA1c] ───────────────────> [Diabetic Retinopathy]     |
| 2. REVERSE CAUSALITY:     [Increased Palliative Consults] <───────── [Severe Terminal Illness]   |
| 3. CONFOUNDING / SPURIOUS: [Hospital Bed Capacity]                   [Post-Op Inpatient Deaths]   |
|                                    ▲                                            ▲                 |
|                                    └──────────── [Hospital Trauma Level] ───────┘                 |
|                                                (Omitted Confounder)                               |
+---------------------------------------------------------------------------------------------------+

Pearson Product-Moment Correlation Coefficient ($r$)

Pearson's $r$ is a parametric measure of the strength and direction of a linear relationship between two continuous, normally distributed variables.

$\text{Pearson's } r = \frac{\sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum_{i=1}^n (x_i - \bar{x})^2 \sum_{i=1}^n (y_i - \bar{y})^2}} = \frac{\text{Cov}(X, Y)}{s_x s_y}$

Parametric Assumptions Required for Pearson's $r$:

1. Continuous Scale: Both variables $X$ and $Y$ must be measured on an interval or ratio scale (e.g., patient age in years and systolic blood pressure in mmHg).
2. Bivariate Normality: Both variables must follow approximately normal distributions in the target population.
3. Linearity: The underlying relationship between $X$ and $Y$ must be straight-line linear, not curvilinear or exponential.
4. Absence of Severe Outliers: Pearson's $r$ is exceptionally sensitive to extreme outlier data points, which can artificially inflate or attenuate the correlation coefficient.
5. Homoscedasticity: The variability of $Y$ values must remain relatively constant across all values of $X$.

Spearman Rank-Order Correlation Coefficient ($\rho$ or $r_s$)

Spearman's $\rho$ is a non-parametric statistic that measures the strength and direction of a monotonic relationship between two variables. It converts raw continuous or ordinal values into numerical ranks ($1, 2, \dots, n$) before computing the correlation.

$\text{Spearman's } \rho = 1 - \frac{6 \sum_{i=1}^n d_i^2}{n(n^2 - 1)}$

where $d_i = \text{Rank}(x_i) - \text{Rank}(y_i)$ represents the difference between the ranks assigned to observation $i$, and $n$ is the total sample size.

Methodological Advantages and Healthcare Applications:

• Monotonic Non-Linear Associations: Captures relationships where variables move in the same relative direction without requiring a strict constant rate of change (e.g., patient age vs. annual emergency department visit frequency).
• Ordinal Data: Specifically suited for ranked healthcare scales such as Cancer Staging (Stage I, II, III, IV), NYHA Congestive Heart Failure Functional Classes (Class I–IV), or Likert patient satisfaction survey responses (1 to 5).
• Robustness to Skewness and Outliers: Because values are converted to ranks, extreme financial outliers (e.g., a $1.2M burn ICU admission) exert no more leverage than the next highest ranked admission.

Quantifying Correlation Strength

Correlation coefficients range strictly between $-1.00$ and $+1.00$:

• $+1.00$: Perfect positive linear/monotonic association.
• $0.00$: Total absence of linear/monotonic association.
• $-1.00$: Perfect negative (inverse) linear/monotonic association.
• Standard Clinical Benchmarks: $|r| < 0.30$ = Weak; $0.30 \le |r| \le 0.69$ = Moderate; $|r| \ge 0.70$ = Strong to Very Strong.

---

2. Simple Linear Regression Analysis

While correlation describes the bidirectional association between two variables without distinguishing cause and effect, Simple Linear Regression models the directional mathematical dependency of a continuous dependent outcome ($Y$) as a linear function of an independent predictor ($X$).

+---------------------------------------------------------------------------------------------------+
|                               SIMPLE LINEAR REGRESSION ARCHITECTURE                               |
+---------------------------------------------------------------------------------------------------+
                                 Y (Dependent Outcome, e.g., LOS)
                                 │                 * (Observed Point: y_i)
                                 │                /│
                                 │               / │ Residual: e_i = y_i - \hat{y}_i
                                 │              /  ▼
                                 │             /  * (Fitted Value: \hat{y}_i)
                                 │            / ┌─────── Slope (\beta_1): \Delta Y / \Delta X
                                 │           /  │
                                 │  *       /   │
                                 │  │      /    ▼
                                 │  └──*  /   * ──────────────────────────────────────────
                                 │       /       
                                 │      /  Regression Line: \hat{Y} = \beta_0 + \beta_1 X
                                 │*    / 
                                 │    /
             Intercept (\beta_0) ───> *   
                                 └─────────────────────────────────────── X (Predictor, e.g., Age)

Mathematical Formulation

The theoretical population simple linear regression model is expressed as:

$Y_i = \beta_0 + \beta_1 X_i + \epsilon_i$

where:

• $Y_i$ is the observed value of the dependent outcome for subject $i$.
• $X_i$ is the observed value of the independent predictor for subject $i$.
• $\beta_0$ is the population $y$-intercept (the expected value of $Y$ when $X = 0$).
• $\beta_1$ is the population slope coefficient (the expected change in $Y$ per 1-unit increase in $X$).
• $\epsilon_i$ is the unobserved random error term, assumed to be independently and identically normally distributed with mean zero and constant variance: $\epsilon_i \sim N(0, \sigma^2)$.

Ordinary Least Squares (OLS) Estimation

The sample regression equation is written as:

$\hat{Y}_i = b_0 + b_1 X_i$

OLS calculates sample parameters $b_0$ (or $\hat{\beta}_0$) and $b_1$ (or $\hat{\beta}_1$) by minimizing the Sum of Squared Residuals (SSE), also called the Sum of Squared Errors:

$\text{SSE} = \sum_{i=1}^n e_i^2 = \sum_{i=1}^n (y_i - \hat{y}_i)^2 = \sum_{i=1}^n [y_i - (b_0 + b_1 x_i)]^2$

Mathematical Derivations for Slope and Intercept:

$b_1 = \frac{\sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^n (x_i - \bar{x})^2} = \frac{\text{Cov}(X,Y)}{s_x^2} = r \left( \frac{s_y}{s_x} \right)$

$b_0 = \bar{y} - b_1 \bar{x}$

Partitioning of Variance & The Coefficient of Determination ($R^2$)

Total variation in the dependent variable is partitioned into explained and unexplained components:

$\text{Total Sum of Squares (SST)} = \text{Regression Sum of Squares (SSR)} + \text{Error Sum of Squares (SSE)}$

$\sum_{i=1}^n (y_i - \bar{y})^2 = \sum_{i=1}^n (\hat{y}_i - \bar{y})^2 + \sum_{i=1}^n (y_i - \hat{y}_i)^2$

$\text{Coefficient of Determination } (R^2) = \frac{\text{SSR}}{\text{SST}} = 1 - \frac{\text{SSE}}{\text{SST}}$

• Interpretation: $R^2$ represents the exact percentage of total variation in the dependent variable ($Y$) that is explained by its linear relationship with the independent predictor ($X$). In simple linear regression, $R^2 = (r)^2$.
• Standard Error of the Estimate ($s_e$ or $S_{yx}$): Measures the standard deviation of the observed data points around the fitted regression line:

$s_e = \sqrt{\frac{\text{SSE}}{n - 2}} = \sqrt{\frac{\sum (y_i - \hat{y}_i)^2}{n - 2}}$

---

3. Multiple Linear Regression & Model Diagnostics

In clinical and operational settings, single-variable models oversimplify reality. Multiple Linear Regression (MLR) extends OLS modeling to accommodate $k$ independent predictors simultaneously, allowing analysts to isolate the unique partial effect of each predictor while mathematically controlling for confounding covariates.

$Y_i = \beta_0 + \beta_1 X_{1i} + \beta_2 X_{2i} + \beta_3 X_{3i} + \dots + \beta_k X_{ki} + \epsilon_i$

Slope Interpretation in MLR (Partial Regression Coefficients)

Each coefficient $\beta_j$ represents the estimated change in the dependent variable $Y$ associated with a 1-unit increase in predictor $X_j$, holding all other independent variables in the model constant (ceteris paribus).

Adjusted Coefficient of Determination ($R^2_{\text{adj}}$)

In multiple regression, adding any additional variable—even pure random noise—will mathematically force the unadjusted $R^2$ to increase or remain unchanged because SSE cannot increase. To prevent model over-fitting and penalize the inclusion of superfluous variables, health data analysts utilize Adjusted $R^2$:

$R^2_{\text{adj}} = 1 - \left[ \frac{(1 - R^2)(n - 1)}{n - k - 1} \right] = 1 - \left( \frac{\text{SSE} / (n - k - 1)}{\text{SST} / (n - 1)} \right)$

where $n$ is total sample size and $k$ is the number of independent predictor variables. $R^2_{\text{adj}}$ will decrease if an added predictor contributes less explanatory power than would be expected by pure chance.

The Classical OLS Assumptions (LINE Framework)

For OLS regression estimates to be Best Linear Unbiased Estimates (BLUE under the Gauss-Markov Theorem), four fundamental assumptions must be satisfied:

+---------------------------------------------------------------------------------------------------+
|                             THE OLS REGRESSION "LINE" ASSUMPTIONS                                 |
+-------------------+-----------------------------------+-------------------------------------------+
| ASSUMPTION        | MATHEMATICAL DEFINITION           | DETECTION & DIAGNOSTIC METHOD             |
+-------------------+-----------------------------------+-------------------------------------------+
| L - Linearity     | E(Y|X) is linear in parameters    | Residuals vs. Fitted Values Scatterplot   |
| I - Independence  | Cov(e_i, e_j) = 0 for all i != j  | Durbin-Watson Test (Target: d ~ 2.0)      |
| N - Normality     | e_i ~ N(0, \sigma^2)              | Q-Q Plot, Shapiro-Wilk, Kolmogorov-Smirnov|
| E - Equal Variance| Var(e_i) = \sigma^2 (Homoscedastic)| Breusch-Pagan Test, White Test, Residuals |
+-------------------+-----------------------------------+-------------------------------------------+

1. Linearity: The relationship between predictors and the mean of the dependent variable is linear. Assessed by plotting residuals against fitted values; curved or parabolic residual patterns indicate non-linearity requiring polynomial terms or log transformations.
2. Independence of Residuals: Residuals are uncorrelated across observations. In longitudinal or time-series health data (e.g., daily ED census), autocorrelation frequently violates this assumption, requiring autoregressive modeling.
3. Normality of Residuals: Residual errors follow a normal distribution with mean zero. Mild violations are tolerated in large samples ($n > 500$) via the Central Limit Theorem, but severe skewness distorts $p$-values and confidence intervals.
4. Equal Variance (Homoscedasticity): The variance of the residuals is constant across all predicted values. Heteroscedasticity occurs when residual spread expands (e.g., residual variance increases as hospital cost increases), which causes standard errors to be underestimated and inflates Type I error rates. Corrected via weighted least squares or Huber-White robust standard errors.

Multicollinearity Detection and Remediation

Multicollinearity occurs when two or more independent variables in a multiple regression model are highly linearly correlated with each other (e.g., including both Patient Weight in kg and BMI, or Systolic BP and Mean Arterial Pressure).

• Consequences: Multicollinearity does not bias the overall model predictions or reduce overall $R^2$, but it severely inflates the standard errors of the regression coefficients ($SE(b_j)$), making individual predictors appear statistically insignificant ($p > 0.05$) even when strong relationships exist.
• Detection via Variance Inflation Factor (VIF): For each predictor $X_j$, regressing $X_j$ on all other remaining $k-1$ predictors yields an individual determination coefficient $R^2_j$:

$\text{VIF}_j = \frac{1}{1 - R^2_j} = \frac{1}{\text{Tolerance}_j}$

$\text{Tolerance}_j = 1 - R^2_j$

• Interpretation Rules:
  • $\text{VIF} = 1.0$: Complete absence of collinearity.
  • $1.0 < \text{VIF} < 5.0$: Mild, acceptable collinearity.
  • $\text{VIF} \ge 5.0$: Concerning collinearity warranting investigation.
  • $\text{VIF} \ge 10.0$: Severe multicollinearity; the affected variable must be removed, combined into a composite index, or analyzed via penalized regression (Ridge / Lasso).

---

4. Logistic Regression for Binary Clinical Outcomes

In healthcare analytics, the most critical clinical endpoints are binary (dichotomous) qualitative events:

• Patient Inpatient Mortality ($1 = \text{Deceased}, 0 = \text{Discharged Alive}$)
• 30-Day Hospital Readmission ($1 = \text{Readmitted}, 0 = \text{No Readmission}$)
• Healthcare-Associated Infection ($1 = \text{Infection Present}, 0 = \text{No Infection}$)
• Claims Fraud / Billing Abuse ($1 = \text{Non-Compliant}, 0 = \text{Compliant}$)

Failure of Linear Probability Models (OLS) for Binary Data

Applying standard linear regression ($Y = \beta_0 + \beta_1 X$) to binary 0/1 outcomes fails for three fundamental mathematical reasons:

1. Nonsensical Predictions: OLS regression lines extend to infinity, producing predicted probabilities $\hat{Y} < 0.00$ or $\hat{Y} > 1.00$.
2. Non-Normal Error Terms: Because $Y$ takes only values 0 or 1, residuals $\epsilon_i = Y_i - \hat{Y}_i$ follow a discrete binomial distribution, violating the OLS normality assumption.
3. Severe Inherent Heteroscedasticity: The variance of a binary variable is $\text{Var}(Y|X) = p(1-p)$. Because $p$ varies as $X$ changes, error variance is mathematically non-constant across predictions.

The Logit Link Function and Logistic Curve

Logistic regression resolves these failures by transforming the non-linear probability curve ($S$-shaped sigmoid curve bounded strictly between 0 and 1) into a linear function using the Logit Transformation (the natural logarithm of the odds).

+---------------------------------------------------------------------------------------------------+
|                                 LOGISTIC REGRESSION TRANSFORMATION                                |
+---------------------------------------------------------------------------------------------------+
                                 p = Probability of Event (0 to 1)
                                 │                       /""""""""""" (Asymptote at p = 1.0)
                                 │                     / 
                                 │                   /   Sigmoid Curve: p = 1 / (1 + e^-z)
                                 │                  /    where z = \beta_0 + \beta_1 X
                                 │                 / 
                                 │                │
                                 │               / 
                                 │              / 
                                 │            /   
                                 │___________/_________________ (Asymptote at p = 0.0)
                                 └─────────────────────────────── Linear Predictor z
                                                │
                                                ▼ (Logit Inversion)
                                   ln(p / (1 - p)) = \beta_0 + \beta_1 X

Mathematical Derivation of the Logistic Model:

Let $p = P(Y = 1 \mid X)$ be the probability that the clinical event occurs.

1. Odds of the Event: $\text{Odds} = \frac{p}{1 - p}$
2. Logit (Log-Odds): $\text{logit}(p) = \ln\left( \frac{p}{1 - p} \right) = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \dots + \beta_k X_k$
3. Inverting to Probability ($p$): $p = \frac{e^{\beta_0 + \sum \beta_j X_j}}{1 + e^{\beta_0 + \sum \beta_j X_j}} = \frac{1}{1 + e^{-(\beta_0 + \sum \beta_j X_j)}}$

Parameter Estimation: Maximum Likelihood Estimation (MLE)

Because OLS cannot be applied to logit models, logistic regression parameters ($\beta_0, \beta_1, \dots, \beta_k$) are estimated using Maximum Likelihood Estimation (MLE). MLE uses iterative numerical optimization (such as the Newton-Raphson algorithm) to find coefficient values that maximize the Log-Likelihood ($LL$) function—the mathematical probability of observing the specific pattern of 0s and 1s present in the sample cohort.

Exponentiated Coefficients: Calculating and Interpreting Odds Ratios ($OR$)

In logistic regression, the raw slope coefficient $\beta_j$ represents the change in the log-odds of the outcome per 1-unit increase in $X_j$. Because humans cannot intuitively interpret log-odds, analysts exponentiate the coefficient to obtain the Odds Ratio ($OR$):

$\text{Odds Ratio } (OR_j) = e^{\beta_j} = \exp(\beta_j)$

• $OR = 1.00$: The predictor has no effect on the odds of the outcome (equivalent to $\beta = 0$).
• $OR > 1.00$: The predictor is associated with increased odds of the outcome. Specifically, $(OR - 1) \times 100%$ represents the percentage increase in odds per 1-unit increase in $X$.
• $OR < 1.00$: The predictor is associated with decreased odds (protective factor). Specifically, $(1 - OR) \times 100%$ represents the percentage decrease in odds per 1-unit increase in $X$.

Constructing 95% Confidence Intervals for Odds Ratios:

Because the sampling distribution of $OR$ is skewed, the 95% CI must be constructed on the logit scale using the standard error $SE(\beta_j)$, and then exponentiated:

$95\% \text{ CI for } OR_j = \left[ e^{\beta_j - 1.96 \cdot SE(\beta_j)}, \; e^{\beta_j + 1.96 \cdot SE(\beta_j)} \right]$

CHDA Exam Rule: If the 95% Confidence Interval for an Odds Ratio spans the null value of $1.00$ (e.g., $95% \text{ CI: } [0.88, 1.42]$), the predictor is not statistically significant at the $\alpha = 0.05$ level.

Model Fit & Goodness-of-Fit Calibration in Logistic Regression

1. Hosmer-Lemeshow Goodness-of-Fit Test:
   • Partitions the sample into deciles of predicted risk ($10$ risk groups from lowest to highest probability).
   • Compares observed event counts against expected model-predicted event counts across all deciles using a Pearson chi-square statistic with $df = 8$.
   • Interpretation: A non-significant $p$-value ($p > 0.05$) indicates good model fit (calibration), meaning expected events do not significantly diverge from observed events. A statistically significant result ($p < 0.05$) indicates poor calibration.
2. Pseudo-$R^2$ Metrics: Because OLS $R^2$ cannot be computed in MLE, surrogate measures approximate variance explanation:
   • McFadden's Pseudo-$R^2$: $R^2_{\text{McFadden}} = 1 - \frac{\ln L_{\text{Full}}}{\ln L_{\text{Null}}}$. Values between $0.20$ and $0.40$ indicate excellent model fit.
   • Cox & Snell $R^2$: Based on log-likelihood ratio, but has a theoretical maximum value $< 1.0$.
   • Nagelkerke $R^2$: Adjusts Cox & Snell to span the complete range from $0.00$ to $1.00$.
3. Information Criteria (AIC and BIC):
   • $\text{AIC} = -2\ln L + 2k$
   • $\text{BIC} = -2\ln L + k\ln(n)$
   • Used to compare competing non-nested models; lower numerical values indicate superior model parsimony and fit.

---

5. Step-by-Step Worked Clinical Calculation: Logistic Regression

Clinical Scenario: A health system data analyst builds a multivariable logistic regression model to predict 30-day all-cause readmission among heart failure patients ($1 = \text{Readmitted}, 0 = \text{Not Readmitted}$). The estimated model coefficients are:

• Intercept ($\beta_0$): $-2.500$
• Diabetes Diagnosis ($X_1$: $1=\text{Yes}, 0=\text{No}$): $\beta_1 = +0.693$, with $SE(\beta_1) = 0.150$
• Patient Age in Years ($X_2$): $\beta_2 = +0.020$, with $SE(\beta_2) = 0.005$
• Prior Hospitalization in Past 6 Months ($X_3$: $1=\text{Yes}, 0=\text{No}$): $\beta_3 = +0.850$, with $SE(\beta_3) = 0.180$

Step 1: Calculate and Interpret the Odds Ratio for Diabetes ($X_1$)

$OR_{\text{Diabetes}} = e^{\beta_1} = e^{0.693} = 2.000$

• Clinical Interpretation: Diabetic heart failure patients have 2.00 times the odds (or a 100% increase in odds) of 30-day hospital readmission compared to non-diabetic patients, holding age and prior hospitalizations constant.

Step 2: Construct the 95% Confidence Interval for the Diabetes Odds Ratio

1. Calculate logit error bounds: $0.693 \pm 1.96(0.150) = 0.693 \pm 0.294 = [0.399, ; 0.987]$
2. Exponentiate boundaries: $[e^{0.399}, ; e^{0.987}] = [1.490, ; 2.683]$

• Conclusion: We are 95% confident that the true population odds ratio of readmission for diabetic patients lies between 1.49 and 2.68. Because the interval does not include $1.00$, the effect is statistically significant at $\alpha = 0.05$.

Step 3: Compute Predicted Readmission Probability for a Specific Patient

Patient Profile: A 70-year-old patient with Diabetes ($X_1 = 1$), Age 70 ($X_2 = 70$), and No Prior Hospitalizations ($X_3 = 0$).

1. Calculate linear predictor ($z$): $z = \beta_0 + \beta_1(1) + \beta_2(70) + \beta_3(0)$ $z = -2.500 + 0.693(1) + 0.020(70) + 0.850(0) = -2.500 + 0.693 + 1.400 = -0.407$
2. Calculate predicted probability ($p$): $p = \frac{1}{1 + e^{-z}} = \frac{1}{1 + e^{-(-0.407)}} = \frac{1}{1 + e^{0.407}} = \frac{1}{1 + 1.5023} = \frac{1}{2.5023} \approx 0.3996 \implies \mathbf{39.96\%}$

• Conclusion: The model estimates that this specific patient has a 39.96% probability of 30-day hospital readmission.

---

6. Master Comparison Table of Statistical Regression Models