4.1 Exponential Dispersion Family & Link Functions

Theoretical Foundations of Generalized Linear Models

For decades, traditional insurance ratemaking relied on one-way rate indications and minimum-bias procedures (such as Bailey-Simon or Bailey's method) to classify risk. The advent of Generalized Linear Models (GLMs)—introduced by Nelder and Wedderburn in 1972 and expanded by McCullagh and Nelder in 1989—revolutionized actuarial pricing. GLMs provided property and casualty (P&C) actuaries with a unified, rigorous statistical framework capable of handling correlated rating variables, non-normal loss distributions, and exposure weights simultaneously.

Classical Ordinary Least Squares (OLS) regression relies on several foundational assumptions that are fundamentally violated by insurance claims data:

1. Normality of errors: Claim counts are discrete non-negative integers ($0, 1, 2, \dots$), and claim severities are strictly positive, highly skewed continuous amounts ($y > 0$).
2. Homoscedasticity (constant variance): In insurance data, risk volatility invariably increases with the expected loss. A policyholder with an expected annual loss of $5,000 exhibits far greater dollar variance than a policyholder with an expected loss of $200$.
3. Additive relationship: OLS models additive effects ($\mu = \mathbf{x}^T \boldsymbol{\beta}$), which can easily predict negative claim frequencies or negative loss amounts when linear predictors are negative—an actuarial impossibility.

GLMs resolve each of these structural limitations by expanding linear models in two directions: allowing the response variable to follow any distribution within the Exponential Dispersion Family (EDF), and connecting the linear predictor to the expected value via a monotonic link function.

---

The Exponential Dispersion Family (EDF)

A continuous or discrete random variable $Y$ belongs to the Exponential Dispersion Family if its probability density function (or probability mass function) can be expressed in the general form:

$f(y; \theta, \phi) = \exp\left( \frac{y\theta - b(\theta)}{a(\phi)} + c(y, \phi) \right)$

Where:

• $\theta$ is the natural (or canonical) parameter, representing location.
• $\phi$ is the dispersion parameter, representing scale or residual variance.
• $a(\phi)$ is the dispersion function, commonly parameterized as $a(\phi) = \frac{\phi}{\omega}$, where $\omega$ is a known prior weight (such as policy earned exposure or claim count).
• $b(\theta)$ is the cumulant function (also known as the log-partition function). It is a strictly convex, twice-differentiable function that fully determines the probability distribution's moments.
• $c(y, \phi)$ is the normalizing function, which depends only on the observed data $y$ and the dispersion parameter $\phi$, ensuring that the density integrates or sums to 1.

Derivation of Mean and Variance from the Cumulant Function

The fundamental elegance of the EDF lies in the direct mathematical relationship between derivatives of the cumulant function $b(\theta)$ and the moments of $Y$.

The log-likelihood for a single observation is given by:

$\ln f(y; \theta, \phi) = \frac{y\theta - b(\theta)}{a(\phi)} + c(y, \phi)$

Taking the first derivative with respect to the natural parameter $\theta$ yields the score function $U(\theta)$:

$U(\theta) = \frac{\partial \ln f}{\partial \theta} = \frac{y - b'(\theta)}{a(\phi)}$

By Bartlett's first identity, the expected value of the score function is identically zero ($\mathbb{E}[U(\theta)] = 0$):

$\mathbb{E}\left[ \frac{Y - b'(\theta)}{a(\phi)} \right] = 0 \implies \mathbb{E}[Y] = b'(\theta)$

Thus, the mean $\mu = \mathbb{E}[Y]$ is simply the first derivative of the cumulant function:

$\mu = b'(\theta)$

To find the variance, we differentiate the score function a second time with respect to $\theta$:

$\frac{\partial^2 \ln f}{\partial \theta^2} = -\frac{b''(\theta)}{a(\phi)}$

By Bartlett's second identity, the variance of the score function equals the negative expectation of the second derivative of the log-likelihood (the Fisher Information):

$\text{Var}(U(\theta)) = -\mathbb{E}\left[ \frac{\partial^2 \ln f}{\partial \theta^2} \right] = \frac{b''(\theta)}{a(\phi)}$

Substituting the definition of the score function $U(\theta) = \frac{Y - \mu}{a(\phi)}$ into the variance expression:

$\text{Var}\left( \frac{Y - \mu}{a(\phi)} \right) = \frac{\text{Var}(Y)}{[a(\phi)]^2} = \frac{b''(\theta)}{a(\phi)} \implies \text{Var}(Y) = a(\phi) b''(\theta)$

Because $b'(\theta) = \mu$ and $b(\theta)$ is strictly convex ($b''(\theta) > 0$), the function $b'(\theta)$ is strictly monotonic and invertible. We can express $\theta$ as a function of the mean: $\theta = (b')^{-1}(\mu)$. Substituting this into $b''(\theta)$ yields the variance function $V(\mu)$:

$V(\mu) = b''(\theta(\mu))$

Therefore, the variance of any EDF distribution decomposes neatly into two distinct terms:

$\text{Var}(Y) = a(\phi) V(\mu) = \frac{\phi}{\omega} V(\mu)$

1. The dispersion component $a(\phi) = \phi / \omega$, which captures scale and sample weighting.
2. The variance function $V(\mu)$, which dictates how the variance responds to changes in the expected mean $\mu$.

---

Core EDF Distributions in Property & Casualty Actuarial Practice

Every distribution used in standard P&C ratemaking belongs to the Exponential Dispersion Family. The specific mathematical form of the variance function $V(\mu)$ uniquely determines the underlying probability distribution.

Mathematical Verification of Key Distributions

1. Poisson Distribution (Claim Frequency)

The Poisson probability mass function is $P(Y = y) = \frac{\mu^y e^{-\mu}}{y!}$. Rewriting this in exponential form:

$f(y; \mu) = \exp\{ y \ln(\mu) - \mu - \ln(y!) \}$

Comparing this with the EDF density:

• Natural parameter: $\theta = \ln(\mu) \implies \mu = e^\theta$
• Cumulant function: $b(\theta) = \mu = e^\theta$
• Dispersion function: $a(\phi) = 1$ (dispersion parameter $\phi = 1$ is fixed)
• Normalizing function: $c(y, \phi) = -\ln(y!)$

Checking moments:

• Mean: $\mu = b'(\theta) = \frac{d}{d\theta}(e^\theta) = e^\theta = \mu$
• Variance function: $V(\mu) = b''(\theta) = e^\theta = \mu$
• Variance: $\text{Var}(Y) = a(\phi) V(\mu) = 1 \cdot \mu = \mu$

This confirms the defining property of the Poisson distribution: the variance strictly equals the mean (equidispersion).

2. Gamma Distribution (Claim Severity)

The Gamma probability density function with shape $\alpha$ and rate $\beta$ (where $\mu = \alpha / \beta$ and variance $\sigma^2 = \alpha / \beta^2 = \mu^2 / \alpha$) can be written with dispersion $\phi = 1/\alpha$:

$f(y; \mu, \phi) = \frac{1}{\Gamma(1/\phi)} \left( \frac{1}{\phi \mu} \right)^{1/\phi} y^{1/\phi - 1} \exp\left( -\frac{y}{\phi \mu} \right)$

In EDF form:

$f(y; \mu, \phi) = \exp\left( \frac{y(-1/\mu) - \ln(\mu)}{\phi} + c(y, \phi) \right)$

• Natural parameter: $\theta = -\frac{1}{\mu} \implies \mu = -\frac{1}{\theta}$
• Cumulant function: $b(\theta) = \ln(\mu) = -\ln(-\theta)$
• Dispersion function: $a(\phi) = \phi = \frac{1}{\alpha}$

Checking moments:

• Mean: $\mu = b'(\theta) = -\frac{1}{-\theta}(-1) = -\frac{1}{\theta} = \mu$
• Variance function: $V(\mu) = b''(\theta) = \frac{1}{\theta^2} = \left(-\frac{1}{\mu}\right)^{-2} = \mu^2$
• Variance: $\text{Var}(Y) = a(\phi) V(\mu) = \phi \mu^2$

Because the variance is proportional to $\mu^2$, the coefficient of variation ($CV$) is constant:

$CV = \frac{\sqrt{\text{Var}(Y)}}{\mu} = \frac{\sqrt{\phi \mu^2}}{\mu} = \sqrt{\phi} = \frac{1}{\sqrt{\alpha}}$

This constant $CV$ property aligns with property damage claims: a $10%$ fluctuation on a commercial property policy is proportionally identical to a $10%$ fluctuation on a personal auto policy, even though the absolute dollar variance is dramatically larger.

---

Link Functions: Canonical vs. Practical

A Generalized Linear Model connects the expected value of the response $\mu_i = \mathbb{E}[Y_i]$ to a linear combination of explanatory rating variables $\mathbf{x}i = [1, x{i1}, x_{i2}, \dots, x_{ik}]^T$ through a monotonic, differentiable link function $g(\cdot)$:

$\eta_i = g(\mu_i) = \mathbf{x}_i^T \boldsymbol{\beta} = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \dots + \beta_k x_{ik}$

Where $\eta_i$ is called the linear predictor. The link function maps the domain of the mean $\mu_i$ (which may be restricted, such as $\mu > 0$ for insurance losses) to the entire real line $(-\infty, \infty)$ of the linear predictor.

The Canonical Link Function

The link function that directly equates the linear predictor to the natural parameter $\theta$ is called the canonical link:

$g(\mu) = \theta(\mu)$

• For the Normal distribution: $\theta = \mu \implies g(\mu) = \mu$ (Identity link).
• For the Poisson distribution: $\theta = \ln(\mu) \implies g(\mu) = \ln(\mu)$ (Log link).
• For the Gamma distribution: $\theta = -1/\mu \implies g(\mu) = -1/\mu$ (or $1/\mu$) (Negative inverse link).
• For the Inverse Gaussian distribution: $\theta = -1/(2\mu^2) \implies g(\mu) = 1/\mu^2$ (Inverse squared link).

Theoretical Properties of Canonical Links

Canonical links offer attractive mathematical properties in statistical estimation:

1. Sufficient statistics: The vector $\mathbf{X}^T \mathbf{y}$ forms a minimal sufficient statistic for $\boldsymbol{\beta}$.
2. Fisher scoring equals Newton-Raphson: The observed Hessian matrix matches the expected Fisher Information matrix everywhere in the parameter space, guaranteeing quadratic convergence.
3. Unbiased marginal totals: Under a canonical link with an intercept, the sum of fitted values strictly equals the sum of observed values across the training data:

$\sum_{i=1}^n y_i = \sum_{i=1}^n \hat{\mu}_i$

Why Canonical Links Often Fail in Practical Actuarial Pricing

Despite their theoretical elegance, canonical links are frequently unviable for insurance pricing outside the Poisson distribution. Consider modeling claim severity using a Gamma distribution with its canonical link $g(\mu) = -1/\mu$ (or $1/\mu$):

$\frac{1}{\mu_i} = \beta_0 + \beta_1 x_{i1} + \dots + \beta_k x_{ik}$

If the linear combination $\mathbf{x}_i^T \boldsymbol{\beta}$ evaluates to a negative number or zero for certain risk profiles (e.g., highly favorable credit tier, high deductible, mature driver), the model predicts a negative or infinite expected severity $\hat{\mu}_i$. Because claim severity is strictly positive ($y > 0$), a negative prediction crashes rating engines and violates the physical reality of insurance claims.

---

Why the Log Link is Universally Favored in P&C Ratemaking

In casualty actuarial practice, both frequency models (Poisson, Negative Binomial) and severity models (Gamma, Inverse Gaussian) are almost universally specified with the log link:

$g(\mu) = \ln(\mu) \iff \mu = \exp(\mathbf{x}^T \boldsymbol{\beta})$

There are two primary reasons why the log link dominates commercial rate filings:

1. Enforcing Strictly Positive Predictions

Because the exponential function maps any real number to the positive real line:

$\mu_i = \exp\left(\beta_0 + \sum_{j=1}^k \beta_j x_{ij}\right) > 0 \quad \forall \quad \mathbf{x}_i^T \boldsymbol{\beta} \in (-\infty, \infty)$

No combination of predictor values—regardless of how extreme or favorable the risk characteristics are—can ever produce a negative or zero frequency, severity, or pure premium.

2. Direct Equivalence to Multiplicative Rating Relativities

Traditional property and casualty rating manuals calculate policyholder premium through a multiplicative algorithm:

$\text{Premium} = \text{Base Rate} \times \text{Territory Factor} \times \text{Driver Class Factor} \times \text{Vehicle Factor}$

The log link translates regression coefficients directly into this multiplicative structure without requiring any ad-hoc adjustments:

$\mu_i = \exp(\beta_0) \times \exp(\beta_1 x_{i1}) \times \exp(\beta_2 x_{i2}) \times \dots \times \exp(\beta_k x_{ik})$

• The intercept term $\exp(\beta_0)$ represents the Base Rate (the expected cost for a policyholder with all baseline attributes).
• For any categorical predictor $x_j$, the exponentiated coefficient $\exp(\beta_j)$ is the Rating Relativity (Rating Factor).
• If a suburban territory has $\beta_{\text{terr}} = 0.20$, its rating factor is $\exp(0.20) = 1.2214$, representing an immediate $+22.14%$ rate relativity relative to the base territory.

GLM Linear Scale (Additive):        eta = beta_0 + beta_1*Territory + beta_2*Class
                                         │
                                    [Log Link: mu = exp(eta)]
                                         │
Actuarial Rating Manual:            mu  = Base Rate * Relativity_Territory * Relativity_Class
                                          [exp(beta_0)]  [exp(beta_1)]           [exp(beta_2)]

---

Retransformation Bias & Jensen's Inequality: GLM vs. Transformed OLS

A classic mistake made by novice analysts is attempting to achieve a multiplicative model by taking the natural logarithm of the response variable and fitting an Ordinary Least Squares regression:

$\ln(Y_i) = \beta_0 + \mathbf{x}_i^T \boldsymbol{\beta} + \varepsilon_i, \quad \varepsilon_i \sim \mathcal{N}(0, \sigma^2)$

Predictions are then obtained by exponentiating: $\hat{Y}_i = \exp(\hat{\beta}_0 + \mathbf{x}_i^T \hat{\boldsymbol{\beta}})$. This procedure suffers from severe retransformation bias governed by Jensen's Inequality.

Because the logarithm is a strictly concave function, the expected value of the log is strictly less than the log of the expected value for any random variable with non-zero variance:

$\mathbb{E}[\ln(Y)] < \ln(\mathbb{E}[Y])$

Consequently, exponentiating the expected log response recovers the geometric mean (or the median under lognormality), not the expected value:

$\exp(\mathbb{E}[\ln(Y)]) \ne \mathbb{E}[Y]$

Under log-normality, the exact relationship is:

$\mathbb{E}[Y] = \exp\left( \mathbb{E}[\ln(Y)] + \frac{\sigma^2}{2} \right) = \exp(\mathbf{x}^T \boldsymbol{\beta}) \cdot \exp\left(\frac{\sigma^2}{2}\right)$

Failing to include the correction factor $\exp(\sigma^2 / 2)$ results in systematic underpricing across the portfolio.

In contrast, a GLM with a log link directly models the logarithm of the expectation:

$\ln(\mathbb{E}[Y]) = \mathbf{x}^T \boldsymbol{\beta} \iff \mathbb{E}[Y] = \exp(\mathbf{x}^T \boldsymbol{\beta})$

The GLM operates on the scale of the original losses, correctly preserving the arithmetic mean without introducing retransformation bias or requiring smearing estimators.

---

Actuarial Exam Traps & Pitfalls

[!WARNING] Trap 1: Confusing the Dispersion Parameter with the Variance Function On CAS exams, candidates often confuse $\phi$ with $V(\mu)$. Remember that the variance is $\text{Var}(Y) = \frac{\phi}{\omega} V(\mu)$. For a Poisson model, $\phi = 1$ and $V(\mu) = \mu$. For a Gamma model, $\phi = 1/\alpha$ (the reciprocal of the shape parameter) and $V(\mu) = \mu^2$. If an exam question asks for the variance function of a Gamma distribution, the correct answer is $\mu^2$, not $\phi \mu^2$.

[!WARNING] Trap 2: Assuming Canonical Links are Mandatory Canonical links possess elegant mathematical properties (e.g., minimal sufficient statistics), but in practice, actuaries almost never use the canonical link for Gamma severity models ($g(\mu) = -1/\mu$). Using the canonical link for Gamma risks negative severity predictions. The log link is standard across the P&C industry for both frequency and severity.

[!WARNING] Trap 3: Log-Transforming the Target in OLS vs. Using a Log-Link GLM Modeling $\ln(Y)$ with OLS models $\mathbb{E}[\ln(Y)]$, which yields the geometric mean when exponentiated. Modeling $Y$ with a log-link GLM models $\ln(\mathbb{E}[Y])$, directly yielding the actuarially required arithmetic mean.