Problem Set 2: Stationary Univariate Processes

Exercise 1

Show that the coefficients of the Wold representation of the stable \(ARMA(1, 1)\) model evolve according to the difference equation

\[\psi_j - \alpha \psi_{j-1} = 0\]

with initial condition \(\psi_1 = \alpha - \beta\).

Without loss of generality we assume \(\delta = 0\) so that the \(ARMA(1, 1)\) representation is given by

\[y_t = \alpha y_{t-1} + \epsilon_t - \beta \epsilon_{t-1}\]

For the Wold representation, the process needs to be converted to an \(MA(\infty)\) process.

\[(1 - \alpha L) y_t = (1 - \beta L) \epsilon_t\]

If all roots of the characteristic equation of \((1 - \alpha L)\) lay outside the unit circle, it is invertible. Furthermore, since \(\alpha < 1\) we can expand the expression to an infinite series.

\[y_t = \frac{1 - \beta L}{1 - \alpha L} \epsilon_t\]

To match the expression of the Wold representation, the following relation has to hold

\[\begin{split}\begin{align} (1 - \beta L) &= (1 - \alpha L) \sum^\infty_{j=0} L^j \psi_j\\ &= \psi_0 & &+ L \psi_1 &&+ L^2 \psi_2 &&+ L^3 \psi_3 &&+ \dots\\ & & &- \alpha L \psi_0 &&- \alpha L^2 \psi_1 &&- \alpha L^3 \psi_2 &&- \dots\\ \end{align}\end{split}\]

Matching by \(L^j\) yields

\[\begin{split}\begin{align} L = 0:&& \psi_0 &= 1\\ L = 1:&& \psi_1 &= \alpha \psi_0 - \beta = \alpha - \beta\\ L = 2:&& \psi_2 &= \alpha \psi_1 = \alpha (\alpha - \beta)\\ L = j:&& \psi_j &= \alpha \psi_{j - 1} \end{align}\end{split}\]

Exercise 2

Consider the following \(ARMA(2, 2)\) process, where \(\epsilon_t \overset{i.i.d.}{\sim} \mathcal{N}(0, 1)\):

\[y_t = 10 + 0.3 y_{t-1} - 0.2 y_{t-2} + \epsilon_t + 0.5 \epsilon_{t-1} + 0.25 \epsilon_{t-2}\]

1. Is the process stationary? Why?

First, the process is rewritten with the lag operator.

\[(1 - 0.3 L + 0.2 L^2) y_t = 10 + (1 + 0.5 L + 0.25 L^2) \epsilon_t\]

The process is stationary if the roots of the characteristic equation of \((1 - 0.3 L + 0.2 L^2)\) lay outside the unit circle. Therefore, we replace \(L\) with \(z\) and solve it using the \(p-q\) formula.

\[\begin{split}\begin{align} 0 &= 1 - 0.3 z + 0.2 z^2\\ 0 &= z^2 - 1.5 z + 5\\ z_{1,2} &= 0.75 \pm \sqrt{0.75^2 - 5}\\ &= 0.75 \pm \sqrt{\frac{9}{16} - \frac{80}{16}}\\ &= 0.75 \pm \frac{\sqrt{71}}{4} \end{align}\end{split}\]

where \(\mid z_{1,2}\mid > 1\) and therefore the process is stationary. The same result could have been obtained by using a different characteristic equation with \(z^{-1} = \lambda\). The characteristic equation can be rewritten to (1)

\[\begin{split}\begin{align} 0 &= 1 - 0.3 z + 0.2 z^2\\ 0 &= 1 - 0.3 \frac{1}{\lambda} + 0.2 \frac{1}{\lambda^2}\\ 0 &= \lambda^2 - 0.3 \lambda + 0.2 \end{align}\end{split}\]

Proceeding with calculating the roots yields

\[\begin{split}\begin{align} 0 &= \lambda^2 - 0.3 \lambda + 0.2\\ \lambda_{1, 2} &= 0.15 \pm \sqrt{0.0225 - 0.2} \end{align}\end{split}\]

where \(\mid\lambda_{1, 2}\mid < 1\).

The reason for the condition on the roots of the equation to be either \(\mid z\mid > 1\) or \(\mid \lambda \mid < 1\) is that

\[ \begin{align}\begin{aligned}:nowrap:\\\dots\end{aligned}\end{align} \]

.

2. Is the process invertible? Why?

The process is invertible if the lag polynomial of the \(MA\) part have characteristic roots being smaller or greater than one depending on the two approaches above. We stick to the lambda representation.

\[\begin{split}\begin{align} 0 &= \lambda^2 + 0.5 \lambda + 0.25\\ \lambda_{1,2} &= -0.25 \pm \sqrt{0.0625 - 0.25} \end{align}\end{split}\]

As the root of a negative number does not exist without imaginery numbers, the roots do not exist and the process is not invertible.

3. Compute \(E(y_t)\).

To compute the expectation of \(y_t\), take the process and stationarity.

\[\begin{split}\begin{align} E[y_t] &= 10 + 0.3 E[y_{t-1}] - 0.2 E[y_{t-2}] + 0 + 0\\ \mu &= 10 + 0.3 \mu - 0.2 \mu\\ \mu &= \frac{100}{9} \end{align}\end{split}\]

4. Compute the autocovariances \(\gamma(\tau)\) for \(\tau = 0, 1, \dots, 4\).

To compute the autocovariances, take the process, multiply \(y_{t-\tau}\) and take expectations.

\[\begin{split}\begin{align} E[y_t y_{t-\tau}] &= E[y_{t-\tau}(10 + 0.3 y_{t-1} - 0.2 y_{t-2} + \epsilon_t + 0.5 \epsilon_{t+1} + 0.25 \epsilon_{t+2}\\ &= 0.3 E[y_{t-\tau} y_{t-1}] - 0.2 E[y_{t-\tau} y_{t-2}] + E[y_{t-\tau} \epsilon_t] + 0.5 E[y_{t-\tau} \epsilon_{t-1}] + 0.25 E[y_{t-\tau} \epsilon_{t-2}] \end{align}\end{split}\]

For the covariances of \(y_{t-\tau}\) and \(\epsilon_i\), the Wold representation is used. Rewrite the process with the lag operator yielding:

\[\begin{align} y_t &= \frac{\delta}{1 - 0.3 + 0.2} + \frac{1 + 0.5L + 0.25 L^2}{1 - 0.3 L + 0.2 L} \epsilon_t \end{align}\]

To suffice the Wold representation, the following equation has to hold.

\[\begin{split}\begin{align} 1 + 0.5 L + 0.25 L^2 &= (1 - 0.3 L + 0.2 L^2) \sum^\infty_{i=0} L^i \psi_i\\ &= \psi_0 &&+ \psi_1 L &&+ \psi_2 L^2 &&+ \psi_3 L^3 &&+ \dots\\ & &&- 0.3 \psi_0 L &&- 0.3 \psi_1 L^2 &&- 0.3 \psi_2 L^3 &&- \dots\\ & && &&+ 0.2 \psi_0 L^2 &&+ 0.2 \psi_1 L^3 &&+ 0.2 \psi_2 L^4 &&+ \dots\\ \end{align}\end{split}\]

Matching by \(L^j\) yields

\[\begin{split}\begin{align} L = 0:&& \psi_0 &= 1\\ L = 1:&& \psi_1 &= 0.5 + 0.3 \psi_0 = 0.8\\ L = 2:&& \psi_2 &= 0.25 + 0.3 \psi_1 - 0.2 \psi_0 = 0.29\\ L = j:&& \psi_j &= 0.3 \psi_{j-1} - 0.2 \psi_{j-2} \end{align}\end{split}\]

Finally, the Wold representation

\[y_t = \frac{100}{9} + (1 + 0.8 L + 0.29 L^2 + \dots) \epsilon_t\]

With the \(ARMA(2, 2)\) represented as \(MA(\infty)\), we can proceed with the autocovariances.

\[\begin{split}\begin{align} \tau = 0:&& \gamma(0) &= 0.3 \gamma(1) - 0.2 \gamma(2) + \sigma^2 + 0.5 * 0.8 \sigma^2 + 0.25 * 0.29 \sigma^2\\ \tau = 1:&& \gamma(1) &= 0.3 \gamma(0) - 0.2 \gamma(1) + 0 + 0.5 \sigma^2 + 0.25 * 0.8 \sigma^2\\ \tau = 2:&& \gamma(2) &= 0.3 \gamma(1) - 0.2 \gamma(0) + 0 + 0 + 0.25 \sigma^2\\ \tau = 3:&& \gamma(3) &= 0.3 \gamma(2) - 0.2 \gamma(1)\\ \tau = 4:&& \gamma(4) &= 0.3 \gamma(3) - 0.2 \gamma(2)\\ \tau = j:&& \gamma(j) &= 0.3 \gamma(j-1) - 0.2 \gamma(j-2) \end{align}\end{split}\]

From the second equation, we get that \(1.2 \gamma(1) = 0.3 \gamma(0) + 0.7 \sigma^2 = \frac{3}{12} \gamma(0) + \frac{7}{12} \sigma^2\). Inserting the expression in the third equation gives

\[\begin{split}\begin{align} \gamma(2) &= 0.3 \gamma(1) - 0.2 \gamma(0) + 0.25 \sigma^2\\ &= 0.3 (\frac{3}{12} \gamma(0) + \frac{7}{12} \sigma^2) - 0.2 \gamma(0) + 0.25 \sigma^2\\ &= \dots \end{align}\end{split}\]

TODO: Finish equations.

5. Compute the coefficients of \(\epsilon_t, \epsilon_{t-1}, \dots, \epsilon_{t-4}\) in the \(MA(\infty)\) representation of \(y_t\).

This can be looked up above in subtask 4.

Exercise 3

Consider the following \(MA(3)\) process, where \(\epsilon_t \overset{i.i.d.}{\sim} \mathcal{N}(0, 1)\):

\[y_t = 10 + \epsilon_t + 0.3 \epsilon_{t-1} + 0.2 \epsilon_{t-2} + 0.4 \epsilon_{t-3}\]

Use Julia to simulate the model and to compute the ACFs and PACFs at orders 1 to 10.

using Distributions, Random, Plots, StatsBase

Random.seed!(123);

dist = Normal()

obs = 1000
lags = 3

err = rand(dist, obs + lags)
y_sim = zeros(obs, 1);

for i = 1:obs
    y_sim[i] = 10 + err[i] + 0.3 * err[i+1] + 0.2 * err[i+2] + 0.4 * err[i+3]
end

plot(y_sim)

autocor_vec = autocor(y_sim, 1:10)
bar(autocor_vec)

pacf_vec = pacf(y_sim, 1:10)
bar(pacf_vec)

Exercise 4

We would like to model the cyclical component of real GDP that we extracted in PS 1.

1. Plot the ACFs and PACFs of the cyclical component. With which model would you start your modeling attempt?
2. Let’s assume we settle on an \(AR(p)\) model. Plot the ACF for the residual series of your model of choice. Is there still autocorrelation present? Run a Ljung-Box Test to verify your result.

using XLSX, DataFrames

gdp = float.(XLSX.readdata("problem_set_1_data/us_real_gdp.xlsx", "FRED Graph", "B12:B295"));

function hodrick_prescott_filter(y, lambda)
    T = size(y, 1)
    matrix = zeros(T, T)

    matrix[1, 1:3] = [1 + lambda, -2 * lambda, lambda]
    matrix[2, 1:4] = [-2 * lambda, 1 + 5 * lambda, -4 * lambda, lambda]

    for i = 3 : T - 2
        matrix[i, i-2 : i+2] = [lambda, -4*lambda, 1 + 6 * lambda, -4 * lambda, lambda]
    end

    matrix[T-1, T-3:T] = [lambda, -4 * lambda, 1 + 5 * lambda, -2 * lambda]
    matrix[T, T-2:T] = [lambda, -2 * lambda, 1 + lambda]

    trend = matrix \ y
    cycle = y - trend

    return trend, cycle
end;

trend, cycle = hodrick_prescott_filter(log.(gdp), 1600);

cycle_acf = autocor(cycle, 1:20)
bar(cycle_acf)

cycle_pacf = pacf(cycle, 1:20)
bar(cycle_pacf)

As the ACFs do not and the PACFs do break-off, we would assume that the underlying process is an \(AR\) process. From the PACF plot we can assume that the process can be modeled with one lagged term.

For the second answer, we assume that an \(AR(1)\) model is sufficient to model the process. First, we need to estimate the model to obtain the fitted values

using GLM, HypothesisTests

cycle_fitted = lm([ones(size(cycle)[1] - 1) cycle[1:end-1]], cycle[2:end])
resid = cycle[2:end] - predict(cycle_fitted);

aic(cycle_fitted)

-1894.6676911705147

bic(cycle_fitted)

-1883.731350477585

resid_acf = autocor(resid, 1:20)
bar(resid_acf)

resid_pacf = pacf(resid, 1:20)
bar(resid_pacf)

LjungBoxTest(resid, 6, 2)

Ljung-Box autocorrelation test
------------------------------
Population details:
    parameter of interest:   autocorrelations up to lag k
    value under h_0:         all zero
    point estimate:          NaN

Test summary:
    outcome with 95% confidence: reject h_0
    one-sided p-value:           <1e-12

Details:
    number of observations:         283
    number of lags:                 6
    degrees of freedom correction:  2
    Q statistic:                    66.53495169569842