6.4 Multiple Regression

Multiple Regression

Markets rarely offer a single driver. Multiple regression writes

Y_i = α + β₁ X_{1i} + β₂ X_{2i} + … + β_k X_{ki} + ε_i

OLS again minimizes Σ ε̂_i², now over α and all β_j. Matrix formulas exist, but FRM focus is interpretation, assumptions, fit measures, and joint tests.

Multiple Versus Single-Regressor Assumptions

Core ideas carry over, with important extensions:

Perfect multicollinearity makes OLS coefficients undefined (singular X′X)—e.g., including both a full set of country dummies and a constant without dropping one category, or including X and 2X. Imperfect multicollinearity (high correlation among X’s) leaves OLS defined but inflates SE(β̂_j), making individual t-tests weak even when the group of variables matters jointly.

Omitted variable bias becomes subtler: if you leave out Z that belongs in the CEF and Z correlates with included X_j, then β̂_j mixes the effect of X_j with the proxy channel through Z.

Coefficient Interpretation: Partial Effects

β̂_j estimates the change in E[Y] associated with a one-unit increase in X_j holding the other included regressors fixed (ceteris paribus within the model).

Example: Y = fund return, X₁ = market, X₂ = size factor. β̂₁ is the market slope after controlling for size exposure—not the same number you get from regressing Y on market alone, unless size is uncorrelated with the market in the sample.

Worked contrast

Simple regression of Y on X₁ alone might give β̃₁ = 1.20. Multiple regression of Y on X₁ and X₂ might give β̂₁ = 0.95, β̂₂ = 0.40.

The drop from 1.20 to 0.95 means part of the raw market association was shared with X₂. Neither coefficient is “wrong”; they answer different questions. FRM items often ask which interpretation matches which regression.

R² Versus Adjusted R²

As before:

TSS = Σ (y_i − ȳ)² ESS = Σ (ŷ_i − ȳ)² RSS = Σ (y_i − ŷ_i)² TSS = ESS + RSS (with intercept) R² = ESS/TSS = 1 − RSS/TSS

Adding a regressor never increases RSS (OLS can always set the new coefficient to zero and recover the old fit). Therefore R² never falls when you add variables—it is a poor tool for choosing models if you only maximize R².

Adjusted R² penalizes complexity:

Adjusted R² = 1 − [(RSS/(n − k − 1)) / (TSS/(n − 1))]

where k is the number of slope parameters (excluding the intercept in the usual counting). Equivalently, compare residual variance estimates with and without the extra parameters. Adjusted R² can fall when the new variable reduces RSS too little to offset the lost degrees of freedom.

Worked numerical sketch

n = 100, k = 2 slopes, RSS = 80, TSS = 200. R² = 1 − 80/200 = 0.60. Adjusted R² = 1 − (80/97)/(200/99) = 1 − (0.8247)/(2.0202) ≈ 1 − 0.408 ≈ 0.592.

Add a third regressor, RSS falls only to 79.5. R² = 1 − 79.5/200 = 0.6025 (up). Adjusted R² = 1 − (79.5/96)/(200/99) ≈ 1 − 0.8281/2.0202 ≈ 1 − 0.410 ≈ 0.590 (down). The variable added fit in-sample by R² but failed the adjusted-R² bar.

Joint Tests (F-Tests)

Individual t-tests ask about one restriction: H₀: β_j = 0. Joint tests ask about several restrictions at once, e.g.

H₀: β₂ = β₃ = 0

(the two extra factors jointly useless) while allowing β₁ free.

Standard F-statistic comparing restricted and unrestricted models:

F = [(RSS_R − RSS_U)/q] / [RSS_U/(n − k_U − 1)]

where q is the number of restrictions, RSS_R is residual SS under H₀, RSS_U under the bigger model. Under classical normal assumptions and H₀, F ~ F_{q, n−k_U−1}.

Large F → the restricted model’s worse fit is unlikely under H₀ → reject the joint null.

Worked example

Unrestricted model (k_U = 3 slopes): RSS_U = 120, n = 84, so df_U = 84 − 3 − 1 = 80. Restricted (drop two regressors, q = 2): RSS_R = 150. F = [(150 − 120)/2] / [120/80] = (15) / (1.5) = 10.

An F of 10 with (2, 80) df far exceeds typical 5% critical values near 3.1 → reject H₀: the two coefficients are jointly zero. Note: each separate t might be only marginally significant when multicollinearity is present, yet the F can still reject—or the opposite pattern can occur depending on correlation structure.

Overall regression F tests H₀: all slopes zero (β₁ = … = β_k = 0). It is the joint test that “the model has no linear explanatory power from the X’s.”

ESS / TSS / RSS Decomposition

Keep the accounting clear:

• TSS: total variation of Y around ȳ.
• ESS: variation of fitted Ŷ around ȳ (what the regressors jointly explain in-sample).
• RSS: leftover variation of residuals.

R² = ESS/TSS. Reporting only R² without checking residual diagnostics, coefficient stability, and economic magnitude is incomplete risk practice—but the decomposition is the language of FRM exam questions on fit.

Worked decomposition

TSS = 500, RSS = 200 → ESS = 300, R² = 300/500 = 0.60. If a competing model posts RSS = 250 on the same Y (same TSS), then ESS = 250, R² = 0.50—worse in-sample linear fit. Whether that model is “better” still depends on out-of-sample performance, theory, and adjusted R² / information criteria—but TSS/ESS/RSS arithmetic must add up.

Putting Multiple Regression to Work in Risk

A desk might regress P&L on market, rates, and FX factors. Partial β̂_j feed risk reporting and hedging ratios conditional on the other factors included. Joint F-tests decide whether a block of style factors belongs in the specification. Adjusted R² discourages kitchen-sink factor lists that inflate plain R². Multicollinearity warns that you may know “the factors matter together” without pinning down each β precisely—use joint tests and economically motivated restrictions rather than hunting every individual p-value.

Exam Checklist

1. Interpret β_j as a partial effect given the other included X’s.
2. State the multiple-regression versions of exogeneity, homo/serial assumptions, and no perfect collinearity.
3. Know R² vs adjusted R² incentives when variables are added.
4. Compute or interpret F from RSS_R, RSS_U, q, and residual df.
5. Tie R² to ESS/TSS/RSS with TSS = ESS + RSS.

Master these and you connect sample moments and hypothesis tests to the multivariate tools used throughout market risk, performance attribution, and empirical FRM applications.