2. Autoregressive and Moving Average Processes

2.1. Moving Average Processes

MA processes are characterized by a mean and lagged white noise. The first order moving average process is

\[\begin{split}x_t &= \mu + u_t - \beta u_{t-1}\\ x_t - \mu &= (1 - \beta L) U-z\end{split}\]

2.2. Autoregressive Processes

AR processes model the dependent variable with weighted lagged terms and white noise. The first order autoregressive process looks like this

\[x_t = \delta + \alpha x_{t-1} + u_t\]

with \(u_t sim \mathcal{N}(0, \sigma^2)\).

The expression can be extended to infinite periods by recursively inserting values for \(x_{t-1}, x_{t-2}, \dots\):

\[x_t = \alpha^{t - t_0} x_{t_0} + (1 + \alpha + \alpha^2 + \dots + \alpha^{t_0}) \delta + \sum^{t - t_0 - 1}_{j=0} \alpha^j u_{t-j}\]

If \(t_0 \to -\infty\) and \(|\alpha| < 1\), the process can be modeled as a constant plus additional white noise.

\[x_t = \underset{E(x_t)}{\frac{\delta}{1 - \alpha}} + \sum^\infty_{j=0} \alpha^j u_{t-j}\]

The equations can also be exressed with the lag operator

\[\begin{split}(1 - \alpha L) x_t &= \delta + u_t\\ x_t &= \frac{\delta}{1 - \alpha L} + \frac{u_t}{1 - \alpha L}\end{split}\]

where \(\frac{1}{1 - \alpha L}\) can be expanded to geometric series

\[\frac{1}{1 - \alpha L} = 1 + \alpha L + \alpha^2 L^2 + \alpha^3 L^3 + \dots\]

2.3. Moments

Properties for \(|\alpha| < 1\):

\[\begin{split}\gamma(0) &= V(x_t)\\ &= E[(x_t - E[x_t])^2]\\ &= E[(\frac{\delta}{1 - \alpha L} + \frac{u_t}{1 - \alpha L} - E[\frac{\delta}{1 - \alpha L} + \frac{u_t}{1 - \alpha L}])^2]\\ &= E[(\frac{\delta}{1 - \alpha L} + \frac{u_t}{1 - \alpha L} - \frac{\delta}{1 - \alpha L} - \frac{E[u_t]}{1 - \alpha L})^2]\\ &= E[(\frac{u_t}{1 - \alpha L})^2]\\ &= E[]\end{split}\]