6.2 Stepwise Selection & Information Criteria Penalties

Automated Subset Selection in Generalized Linear Models

In property and casualty (P&C) ratemaking, selecting an optimal subset of rating predictors is a critical balance between predictive precision, commercial interpretability, and regulatory defensibility. An insurance pricing model must capture legitimate risk differentials without over-parameterizing the rating algorithm or importing sample-specific noise into published rating manuals.

Historically, actuaries relied on automated variable selection procedures to screen candidate variables. Given $p$ candidate rating variables, there are $2^p$ possible distinct subset models. When $p$ is moderate to large (e.g., $p = 40$, yielding $2^{40} \approx 1.1 \times 10^{12}$ candidate models), exhaustive best-subset search is computationally intractable. Automated selection algorithms navigate this combinatorial space via greedy, iterative heuristics.

                         AUTOMATED SUBSET ALGORITHMS
                                      │
       ┌──────────────────────────────┼──────────────────────────────┐
       ▼                              ▼                              ▼
Forward Addition             Backward Elimination           Stepwise Bidirectional
Start: Null Model (β₀)       Start: Full Model (all p)      Start: Null or Intercept
Iteratively ADD variable     Iteratively DROP variable      Alternate ADD and DROP
with largest deviance drop   with smallest deviance drop    Re-test existing variables
Stops when ΔD < threshold    Stops when all p ≤ α_stay      Prevents trapped paths

1. Forward Addition

Forward selection begins with the simplest possible baseline—the null model containing only the intercept parameter $\beta_0$:

$\ln(\mu_i) = \beta_0$

At each subsequent step:

1. The algorithm evaluates every candidate predictor $X_j$ not currently present in the model by fitting a single-parameter addition.
2. It identifies the variable that produces the largest reduction in model deviance (or equivalently, the largest increase in log-likelihood $\Delta \ell$, or the lowest p-value in a score test).
3. If the deviance drop exceeds a predefined threshold (or if the p-value is below a specified entry criterion $\alpha_{\text{enter}}$), the variable is permanently added to the model.
4. The process repeats iteratively until no remaining candidate satisfies the entry criterion.

Actuarial Limitation: Forward addition is myopic. Because variables added in early stages cannot be subsequently removed, the algorithm cannot account for changing covariance structures as additional predictors enter. A variable that appeared highly predictive initially may become redundant once a collinear cluster of variables enters later.

2. Backward Elimination

Backward elimination begins at the opposite extreme—the saturated or full candidate model containing all $p$ candidate predictors simultaneously:

$\ln(\mu_i) = \beta_0 + \sum_{j=1}^p \beta_j X_{ij}$

At each subsequent step:

1. The algorithm assesses the marginal significance of each variable currently in the model using a Wald test, score test, or likelihood ratio test (LRT) drop-in-deviance.
2. The variable that contributes the least to the model—the one whose removal causes the smallest increase in deviance (largest p-value)—is flagged.
3. If this largest p-value exceeds a predefined removal criterion $\alpha_{\text{stay}}$, the variable is eliminated.
4. The process repeats until all variables remaining in the model satisfy the condition $p \le \alpha_{\text{stay}}$.

Actuarial Limitation: Backward elimination requires fitting the full model upfront. If $p > n$ (common in genomics, though rarer in core P&C ratemaking), or if high-dimensional dummy coding creates severe rank-deficiency in the information matrix $\mathbf{X}^T \mathbf{W} \mathbf{X}$, the full model cannot be uniquely estimated by maximum likelihood.

3. Stepwise Bidirectional Selection

Stepwise bidirectional selection combines forward addition and backward elimination. Starting from either the null model or an initial baseline rating structure:

1. At each step, candidate variables not in the model are evaluated for addition using an entry threshold $\alpha_{\text{enter}}$.
2. If a variable enters, all variables currently in the model are immediately re-evaluated against a removal threshold $\alpha_{\text{stay}}$ to determine if any have become redundant.
3. The algorithm alternates between entry and elimination steps until no variables can enter and none can be removed.

                        STEPWISE BIDIRECTIONAL LOOP
                                     │
                       ┌─────────────▼─────────────┐
                       │ Evaluate Unselected Vars  │
                       └─────────────┬─────────────┘
                                     │
                         Best p-value < α_enter?
                                    / \
                              YES  /   \  NO ───► STOP (Final Model)
                                  ▼     
                          Add Variable to Model
                                  │
                       ┌──────────▼──────────┐
                       │  Re-test All Vars   │
                       │  Currently in Model │
                       └──────────┬──────────┘
                                  │
                         Worst p-value > α_stay?
                                 / \
                           YES  /   \  NO
                               ▼     └──► Continue to next cycle
                       Remove Variable
                               │
                               └──► Continue to next cycle

[!IMPORTANT] The Critical Alpha Constraint: To prevent the stepwise algorithm from entering an infinite loop—where the same variable is repeatedly added in the forward phase and removed in the backward phase—the threshold parameters must strictly satisfy: $\alpha_{\text{enter}} < \alpha_{\text{stay}}$ Setting $\alpha_{\text{enter}} = 0.05$ and $\alpha_{\text{stay}} = 0.10$ is a standard convention, ensuring that a variable requires stronger evidence to enter than to remain.

---

Stopping Rules: P-Value Thresholds vs. Information Criteria

Automated selection routines require an objective mathematical stopping rule. Historically, statistical software utilized hypothesis test p-values based on the Likelihood Ratio Test (LRT):

$\Delta D = D_{\text{reduced}} - D_{\text{full}} = 2 \left[ \ell(\hat{\boldsymbol{\beta}}_{\text{full}}) - \ell(\hat{\boldsymbol{\beta}}_{\text{reduced}}) \right] \sim \chi^2_{\Delta df}$

Where $\Delta df$ is the difference in the number of estimated parameters. If the p-value $P(\chi^2_{\Delta df} > \Delta D)$ is below $\alpha_{\text{enter}}$, the variable is retained.

The Breakdown of P-Values on Large P&C Datasets

While p-value stopping rules function acceptably on classical small-sample statistical experiments ($n = 100$ to $1,000$), they fail catastrophically when applied to modern insurance datasets where sample size $n$ regularly exceeds $100,000$ to $10,000,000$ policy records.

The standard error of any estimated regression coefficient scales inversely with the square root of the effective sample size (or earned exposure):

$\text{SE}(\hat{\beta}_j) \propto \frac{1}{\sqrt{n}}$

As $n \to \infty$, $\text{SE}(\hat{\beta}_j) \to 0$. Consequently, the Wald test statistic:

$z = \frac{\hat{\beta}_j - 0}{\text{SE}(\hat{\beta}_j)} = \hat{\beta}_j \sqrt{n} \cdot c$

grows arbitrarily large for any non-zero coefficient, no matter how microscopically trivial the true underlying effect. In a personal auto portfolio with $500,000$ earned car-years:

• A completely irrelevant variable—such as the policyholder's favorite color or the day of the week the application was signed—might exhibit a tiny sample frequency relativity difference of $0.15%$ ($\hat{\beta} = 0.0015$).
• With $n = 500,000$, $\text{SE}(\hat{\beta}) \approx 0.0003$.
• The resulting test statistic is $z = 0.0015 / 0.0003 = 5.0$, yielding $p \approx 0.0000006$.

Under a standard $\alpha_{\text{enter}} = 0.05$ threshold, this completely spurious, commercially absurd variable is aggressively selected. Classical hypothesis testing answers the question: "Is this coefficient exactly zero in the population?" On massive insurance datasets, virtually no physical variable has an effect size of identically zero down to the eighth decimal place. Statistical significance decouples entirely from practical actuarial significance.

---

Methodological Flaws of Stepwise Selection on Actuarial Datasets

Modern actuarial literature (including Frank Harrell's Regression Modeling Strategies and CAS research papers) strongly cautions against stepwise selection. Beyond the sample size dilemma, stepwise algorithms introduce four fatal methodological distortions:

1. Severe Type I Error Inflation

Stepwise selection conducts dozens or hundreds of sequential hypothesis tests without adjusting for multiple comparisons. If an actuary screens 50 candidate telematics variables at nominal $\alpha = 0.05$, the family-wise error rate ($FWER$)—the probability of falsely selecting at least one purely random noise variable—is:

$\text{FWER} = 1 - (1 - \alpha)^m = 1 - (0.95)^{50} \approx 1 - 0.0769 = 92.31\%$

The algorithm is virtually guaranteed to include false positives, mistaking sampling noise for genuine actuarial risk signals.

2. Selection Bias & The Hurdle Effect (Winner's Curse)

When a variable is selected because its test statistic cleared a significance hurdle, its estimated coefficient is systematically biased away from zero:

$\mathbb{E}\left[ |\hat{\beta}_j| \,\Big|\, \text{Variable } X_j \text{ selected} \right] > |\beta_j|$

In finite samples, an estimated coefficient $\hat{\beta}_j$ equals the true effect $\beta_j$ plus random estimation error $\varepsilon_j$. A variable is most likely to clear the entry hurdle when its random noise happens to align in the same direction as its true effect, artificially inflating the test statistic. This phenomenon—the hurdle effect or winner's curse—means that coefficients in stepwise models are systematically exaggerated. When exponentiated into rating relativities $\exp(\hat{\beta}_j)$, the model over-prices higher-risk tiers and under-prices lower-risk tiers.

3. Artificially Narrow Standard Errors & Falsely Narrow Confidence Intervals

Standard statistical software prints a regression summary table displaying standard errors, t-statistics, and confidence intervals. However, the software calculates these quantities under the assumption that the model structure was fixed a priori before looking at the data.

In reality, the degrees of freedom expended during the search across all candidate variables are ignored. The software treats the final $k$ selected variables as if they were the only $k$ variables ever considered, drastically understating the true parameter uncertainty. The reported standard errors are falsely narrow, p-values are deceptively small, and confidence intervals provide an illusion of precision.

4. Extreme Instability Under Collinearity

When candidate rating variables exhibit collinearity (e.g., credit score vs. insurance tier, vehicle horsepower vs. vehicle weight, territory population density vs. vehicle theft rate), stepwise algorithms become erratic. A minor perturbation in the data—such as withholding one cross-validation fold or updating the model with one new quarter of claims—causes the algorithm to select completely different subsets of variables. Such instability makes the resulting rating plan impossible to defend before state insurance regulators or explain to senior underwriting executives.

---

Penalized Likelihood Criteria: AIC vs. BIC

To overcome the limitations of arbitrary p-value thresholds, actuaries utilize information criteria that directly penalize the log-likelihood function for model complexity.

                     PENALIZED LIKELIHOOD CRITERIA
                                   │
            ┌──────────────────────┴──────────────────────┐
            ▼                                             ▼
  Akaike Information (AIC)                      Bayesian Information (BIC)
  Formula: -2·ln(L) + 2k                        Formula: -2·ln(L) + k·ln(n)
  Penalty per Parameter: 2                      Penalty per Parameter: ln(n)
  Objective: Minimize KL divergence             Objective: Maximize posterior prob
  Target: Best out-of-sample prediction         Target: Identify "true" parsimonious model
  Large Datasets: Retains too many variables    Large Datasets: Heavily penalizes complexity

Mathematical Formulation

Let $\ell(\hat{\boldsymbol{\beta}})$ denote the maximized log-likelihood of a GLM, $k$ denote the total number of estimated parameters (including the intercept and any estimated dispersion parameter $\phi$), and $n$ denote the total effective sample size (number of independent policy records or earned exposure units).

$\text{AIC} = -2\ell(\hat{\boldsymbol{\beta}}) + 2k = D(\mathbf{y}, \hat{\boldsymbol{\mu}}) + 2k\phi$

$\text{BIC} = -2\ell(\hat{\boldsymbol{\beta}}) + k \ln(n) = D(\mathbf{y}, \hat{\boldsymbol{\mu}}) + k\ln(n)\phi$

Where $D(\mathbf{y}, \hat{\boldsymbol{\mu}})$ is the unscaled deviance. In both criteria, smaller values indicate superior models. The first term ($-2\ell$) rewards goodness-of-fit, while the second term ($2k$ or $k\ln(n)$) penalizes model complexity.

Theoretical Foundations: Efficiency vs. Consistency

• Akaike Information Criterion (AIC): Derived by Hirotugu Akaike from information theory, AIC provides an asymptotically unbiased estimator of the relative Kullback-Leibler (K-L) divergence between the fitted model and the true unknown data-generating process. AIC is asymptotically efficient: as $n \to \infty$, it selects the model that minimizes mean squared prediction error. However, AIC is not consistent: even as $n \to \infty$, the probability that AIC selects an over-parameterized model containing irrelevant variables does not converge to zero; it approaches a non-zero constant ($P(\text{overfitting}) \approx 15.7%$ for a 1-parameter addition).
• Bayesian Information Criterion (BIC): Derived by Gideon Schwarz from a Bayesian framework, BIC approximates the logarithm of the marginal likelihood (Bayes factor) under a uniform prior over candidate models. BIC is statistically consistent: as $n \to \infty$, the probability that BIC selects the true data-generating model approaches $1.0$ (assuming the true model is in the candidate set).

The Sample Size Multiplier: Why BIC Dominates in Big P&C Data

The fundamental operational difference between AIC and BIC lies in the parameter penalty coefficient:

• AIC Penalty per parameter: Exactly $2.0$, regardless of sample size.
• BIC Penalty per parameter: $\ln(n)$, which grows monotonically with the sample size.

The threshold at which BIC penalizes complexity more harshly than AIC occurs when:

$\ln(n) > 2 \iff n > e^2 \approx 7.389$

For any dataset with more than 7 observations, BIC imposes a steeper penalty than AIC. On large insurance portfolios, this difference becomes massive:

Practical Ratemaking Implications

Consider adding a single rating level (e.g., distinguishing between 4-door sedans and 2-door coupes, adding $\Delta k = 1$) to a personal auto frequency GLM fit on $1,000,000$ earned vehicle-years:

• Under AIC: The variable is added if the deviance drop satisfies $\Delta D > 2.00$. Under a $\chi^2_1$ distribution, a critical value of $2.00$ corresponds to a p-value of $p < 0.1573$. AIC acts as a very loose filter, readily absorbing minor, borderline rating factors that capture minor sample quirks.
• Under BIC: The variable is added only if the deviance drop satisfies $\Delta D > 13.82$. Under a $\chi^2_1$ distribution, a critical value of $13.82$ corresponds to a p-value of $p < 0.0002$. BIC requires overwhelming statistical evidence before permitting an additional parameter.

In casualty insurance ratemaking, parsimony is paramount. Every additional rating tier must be implemented in policy administration systems, monitored in rate filings, and defended before state insurance commissioners. Over-parameterized models lead to unstable rating relativities, thin risk cells, and potential regulatory disapproval. Consequently, actuaries favor BIC (or regularized regression) over AIC when evaluating rating structures on large policy databases.