3.3 Time-Series Analysis: AR Models, Nonstationarity & Cointegration

Time-Series Analysis in Investment Valuation

Financial and macroeconomic data are inherently sequential. Applying standard cross-sectional OLS regression to sequential time series without testing for stationarity leads to invalid statistical inference, parameter instability, and spurious regressions.

---

1. Covariance Stationarity

A time series $\{x_t\}$ is defined as covariance stationary (or weakly stationary) if it satisfies three strict statistical conditions:

1. Constant Expected Value (Mean): The mean of the series is constant and finite over all time periods: $$E(x_t) = \mu \quad \text{for all } t$$
2. Constant Finite Variance: The variance of the series is constant and finite over all time periods: $$Var(x_t) = E[(x_t - \mu)^2] = \sigma^2 < \infty \quad \text{for all } t$$
3. Constant Covariance across Lags: The covariance between observations separated by $s$ periods depends solely on the displacement (lag $s$) and not on the time index $t$: $$Cov(x_t, x_{t-s}) = E[(x_t - \mu)(x_{t-s} - \mu)] = \gamma_s \quad \text{for all } t, s$$

---

2. Autoregressive (AR) Models

An autoregressive model of order $p$, denoted AR($p$), models the current value $x_t$ as a function of its own $p$ lagged values and a white noise disturbance term $\epsilon_t$:

$$x_t = b_0 + b_1 x_{t-1} + b_2 x_{t-2} + \dots + b_p x_{t-p} + \epsilon_t, \quad \epsilon_t \sim WN(0, \sigma^2)$$

The AR(1) Model & The Mean-Reverting Level

For a first-order autoregressive process, AR(1):

$$x_t = b_0 + b_1 x_{t-1} + \epsilon_t$$

• Stationarity Condition: An AR(1) model is covariance stationary if and only if $|b_1| < 1$.
• Mean-Reverting Level ($x_L$): If a series is covariance stationary, setting $E(x_t) = E(x_{t-1}) = x_L$ yields:

$$x_L = b_0 + b_1 x_L \implies x_L(1 - b_1) = b_0 \implies \mathbf{x_L = \frac{b_0}{1 - b_1}}$$

• Economic Meaning: If $x_t > x_L$, the expected value for the next period is less than $x_t$ (reverting downward toward $x_L$). If $x_t < x_L$, the expected value for the next period is greater than $x_t$ (reverting upward toward $x_L$).

Multi-Period Forecasting in AR(1)

Forecasting proceeds recursively using the chain rule of forecasting:

• One-step ahead forecast: $\hat{x}_{t+1} = \hat{b}_0 + \hat{b}_1 x_t$
• Two-step ahead forecast: $\hat{x}_{t+2} = \hat{b}_0 + \hat{b}_1 \hat{x}_{t+1} = \hat{b}_0 + \hat{b}_1 (\hat{b}_0 + \hat{b}_1 x_t) = \hat{b}_0(1 + \hat{b}_1) + \hat{b}_1^2 x_t$
• As the forecast horizon $h \to \infty$, the forecast converges asymptotically to the mean-reverting level $x_L$.

---

3. Residual Diagnostics & Model Selection

Testing for Residual Autocorrelation

A correctly specified AR($p$) model must transform the data such that the estimated residuals $e_t$ behave as white noise (serially uncorrelated). To verify this:

1. Compute the sample autocorrelations of the residuals at lag $k$: $r_k = \text{Corr}(e_t, e_{t-k})$.
2. The standard error of the sample autocorrelation is $s_{r_k} \approx \frac{1}{\sqrt{T}}$, where $T$ is sample size.
3. Compute the test statistic: $t = \frac{r_k}{1/\sqrt{T}} = r_k \sqrt{T}$ with $df = T - p - 1$.
4. If the residual autocorrelation at any lag is statistically significant ($|t| > t_{crit}$), the AR model is underspecified and additional lags must be added.

Moving Average (MA) and ARMA Models

• MA($q$) Model: $x_t = \mu + \epsilon_t + \theta_1 \epsilon_{t-1} + \dots + \theta_q \epsilon_{t-q}$. Autocorrelations drop to zero after lag $q$.
• ARMA($p, q$) Model: Combines $p$ autoregressive terms and $q$ moving average terms: $x_t = b_0 + \sum_{i=1}^p b_i x_{t-i} + \epsilon_t + \sum_{j=1}^q \theta_j \epsilon_{t-j}$.

Information Criteria: AIC vs. BIC (SC)

When comparing competing ARMA models, analysts evaluate information criteria that balance goodness-of-fit against model complexity:

$$\text{Akaike Information Criterion (AIC)} = \ln(\text{MSE}) + \frac{2k}{T}$$ $$\text{Schwarz Bayesian Information Criterion (BIC / SC)} = \ln(\text{MSE}) + \frac{k \ln(T)}{T}$$

Where $k = p + q + 1$ is total estimated parameters. Lower values indicate superior models. BIC imposes a larger penalty on additional parameters than AIC (since $\ln(T) > 2$ for $T \ge 8$), making BIC favor more parsimonious models.

---

4. Nonstationarity: Random Walks & Unit Roots

Random Walk Processes

A time series is a random walk if current value equals previous value plus a random shock:

$$x_t = x_{t-1} + \epsilon_t, \quad \epsilon_t \sim WN(0, \sigma^2)$$

• Here, $b_1 = 1$. The process possesses a unit root.
• Expected value: $E(x_t) = x_0$ (constant).
• Variance: $Var(x_t) = t \sigma^2$. The variance grows infinitely with time $t$, violating covariance stationarity.
• Mean-reverting level: $x_L = \frac{0}{1 - 1} = \text{undefined}$.

Random Walk with Drift

$$x_t = b_0 + x_{t-1} + \epsilon_t, \quad b_0 \neq 0$$

• Expected value: $E(x_t) = x_0 + b_0 t$. The series exhibits a deterministic linear time trend and infinite variance ($t \sigma^2$).

Unit Root Testing: Dickey-Fuller & Augmented Dickey-Fuller (ADF)

We cannot test $b_1 = 1$ using standard $t$-tests because nonstationary series violate the assumptions underlying the Student's $t$-distribution. The Dickey-Fuller (DF) test transforms the AR(1) equation by subtracting $x_{t-1}$ from both sides:

$$x_t - x_{t-1} = b_0 + (b_1 - 1) x_{t-1} + \epsilon_t \implies \Delta x_t = b_0 + g x_{t-1} + \epsilon_t$$

Where $g = b_1 - 1$. The hypotheses are:

• $H_0: g = 0$ ($b_1 = 1 \implies$ Unit root present / Nonstationary)
• $H_a: g < 0$ ($b_1 < 1 \implies$ Covariance stationary)

The test uses specialized Dickey-Fuller critical $\tau$-values (which are substantially more negative than standard normal critical values, e.g., approx $-2.89$ at $\alpha = 0.05$ with an intercept). If $t_{stat} < \tau_{crit}$, reject $H_0$ and conclude the series is stationary.

• Augmented Dickey-Fuller (ADF): Adds lagged changes $\sum_{i=1}^k c_i \Delta x_{t-i}$ to eliminate higher-order residual autocorrelation:

$$\Delta x_t = b_0 + g x_{t-1} + \sum_{i=1}^k c_i \Delta x_{t-i} + \epsilon_t$$

Remedy for Unit Roots: First Differencing

If a time series contains a unit root (integrated of order 1, denoted $x_t \sim I(1)$), taking the first difference produces a stationary series ($y_t \sim I(0)$):

$$y_t = \Delta x_t = x_t - x_{t-1}$$

---

5. Seasonality in Time Series

Quarterly or monthly financial data often exhibit regular seasonal swings (e.g., retail holiday revenue).

• Detection: Seasonality is detected when the sample autocorrelations of residuals spike significantly at the seasonal lag (lag 4 for quarterly data, lag 12 for monthly data).
• Remedy: Add a seasonal lag term directly into the autoregressive equation:

$$x_t = b_0 + b_1 x_{t-1} + b_4 x_{t-4} + \epsilon_t \quad (\text{for quarterly seasonality})$$

---

6. Autoregressive Conditional Heteroskedasticity (ARCH)

In asset return series, periods of high volatility cluster together, as do periods of low volatility (volatility clustering). While return errors $\epsilon_t$ may be serially uncorrelated ($Cov(\epsilon_t, \epsilon_{t-s}) = 0$), their variance is conditional on past squared shocks.

An ARCH(1) model specifies the conditional variance $\sigma_t^2$ as:

$$\sigma_t^2 = a_0 + a_1 \epsilon_{t-1}^2, \quad a_0 > 0, a_1 \ge 0$$

• Testing for ARCH: Estimate the primary AR model, obtain residuals $e_t$, and regress squared residuals on lagged squared residuals:

$$\ne_t^2 = a_0 + a_1 e_{t-1}^2 + u_t$$

Test $H_0: a_1 = 0$ (no ARCH) vs $H_a: a_1 > 0$ using an individual $t$-test or the Engle LM test ($LM = T \times R^2 \sim \chi^2_1$). If $a_1$ is statistically significant, ARCH effects are present.

• Consequences & Forecasting: If ARCH exists, OLS standard errors are invalid. Generalized Autoregressive Conditional Heteroskedasticity (GARCH) models must be estimated, and one-step-ahead volatility is forecasted as $\hat{\sigma}_{t+1}^2 = \hat{a}_0 + \hat{a}_1 e_t^2$.

---

7. Multi-Time-Series Regressions & Cointegration

When regressing one time series $Y_t$ on another $X_t$, the stationarity of both series governs the validity of the regression:

Cointegration & The Engle-Granger Test

Two nonstationary $I(1)$ series are cointegrated if there exists a cointegrating parameter $\beta$ such that their linear combination is stationary ($I(0)$):

$$\ne_t = Y_t - \beta_0 - \beta_1 X_t \sim I(0)$$

• Engle-Granger Two-Step Test:
  1. Run OLS regression in levels: $Y_t = b_0 + b_1 X_t + e_t$.
  2. Test the residual series $e_t$ for a unit root using the Augmented Dickey-Fuller test with specialized Engle-Granger critical values.
  3. If $H_0$ of a unit root in residuals is rejected, the series are cointegrated.
• Error Correction Model (ECM): If cointegrated, the short-run dynamics are modeled with an error-correction mechanism:

$$\Delta Y_t = \alpha_0 + \alpha_1 \Delta X_t + \lambda (Y_{t-1} - \hat{b}_1 X_{t-1}) + u_t$$

Where $\lambda < 0$ is the error correction coefficient measuring the speed of adjustment back to long-run equilibrium.