Time Series Analysis - Moving average and ARMA processes

101 Pages·2012·1.06 MB·English

Time Series Analysis
Autoregressive, MA and ARMA processes
Andr´es M. Alonso Carolina Garc´ıa-Martos
UniversidadCarlosIIIdeMadrid
UniversidadPolit´ecnicadeMadrid
June – July, 2012
AlonsoandGarc´ıa-Martos (UC3M-UPM) TimeSeriesAnalysis June–July,2012 1/50
4. Autoregressive, MA and ARMA processes
4.1 Autoregressive processes
Outline:
Introduction
The first-order autoregressive process, AR(1)
The AR(2) process
The general autoregressive process AR(p)
The partial autocorrelation function
Recommended readings:
(cid:3) Chapter 2 of Brockwell and Davis (1996).
(cid:3) Chapter 3 of Hamilton (1994).
(cid:3) Chapter 3 of Pen˜a, Tiao and Tsay (2001).
AlonsoandGarc´ıa-Martos (UC3M-UPM) TimeSeriesAnalysis June–July,2012 2/50
Introduction
(cid:3) In this section we will begin our study of models for stationary processes which
are useful in representing the dependency of the values of a time series on its past.
(cid:3) The simplest family of these models are the autoregressive, which generalize the
idea of regression to represent the linear dependence between a dependent variable
y (z ) and an explanatory variable x (z ), using the relation:
t t−1
z =c +bz +a
t t−1 t
where c and b are constants to be determined and a are i.i.d N(0,σ2). Above
t
relation define the first order autoregressive process.
(cid:3) This linear dependence can be generalized so that the present value of the
series, z , depends not only on z , but also on the previous p lags, z ,...,z .
t t−1 t−2 t−p
Thus, an autoregressive process of order p is obtained.
AlonsoandGarc´ıa-Martos (UC3M-UPM) TimeSeriesAnalysis June–July,2012 3/50
The first-order autoregressive process, AR(1)
(cid:3) We say that a series z follows a first order autoregressive process, or AR(1),
t
if it has been generated by:
z =c +φz +a (33)
t t−1 t
where c and −1<φ<1 are constants and a is a white noise process with
t
variance σ2. The variables a , which represent the new information that is added
t
to the process at each instant, are known as innovations.
Example 35
We will consider z as the quantity of water at the end of the month in a
t
reservoir. During the month, c +a amount of water comes into the reservoir,
t
where c is the average quantity that enters and a is the innovation, a random
t
variable of zero mean and constant variance that causes this quantity to vary from
one period to the next.
If a fixed proportion of the initial amount is used up each month, (1−φ)z , and
t−1
a proportion, φz , is maintained the quantity of water in the reservoir at the
t−1
end of the month will follow process (33).
AlonsoandGarc´ıa-Martos (UC3M-UPM) TimeSeriesAnalysis June–July,2012 4/50
The first-order autoregressive process, AR(1)
(cid:3) The condition −1<φ<1 is necessary for the process to be stationary. To
prove this, let us assume that the process begins with z =h, with h being any
0
fixed value. The following value will be z =c +φh+a , the next,
1 1
z =c +φz +a =c+ φ(c +φh+a )+a and, substituting successively, we
2 1 2 1 2
can write:
z = c +φh+a
1 1
z = c(1+φ)+φ2h+φa +a
2 1 2
z = c(1+φ+φ2)+φ3h+φ2a +φa +a
3 1 2 3
. . .
. . .
. . .
z = c(cid:80)t−1φi +φth+(cid:80)t−1φia
t i=0 i=0 t−i
If we calculate the expectation of z , as E[a ]=0,
t t
(cid:88)t−1
E[z ]=c φi +φth.
t
i=0
For the process to be stationary it is a necessary condition that this function does
not depend on t.
AlonsoandGarc´ıa-Martos (UC3M-UPM) TimeSeriesAnalysis June–July,2012 5/50
The first-order autoregressive process, AR(1)
(cid:3) The mean is constant if both summands are, which requires that on increasing
t the first term converges to a constant and the second is canceled. Both
conditions are verified if |φ|<1 , because then (cid:80)t−1φi is the sum of an
i=0
geometric progression with ratio φ and converges to c/(1− φ), and the term φt
converges to zero, thus the sum converges to the constant c/(1− φ).
(cid:3) With this condition, after an initial transition period, when t →∞, all the
variables z will have the same expectation, µ=c/(1−φ), independent of the
t
initial conditions.
(cid:3) We also observe that in this process the innovation a is uncorrelated with the
t
previous values of the process, z for positive k since z depends on the
t−k t−k
values of the innovations up to that time, a ,...,a , but not on future values.
1 t−k
Since the innovation is a white noise process, its future values are uncorrelated
with past ones and, therefore, with previous values of the process, z .
t−k
AlonsoandGarc´ıa-Martos (UC3M-UPM) TimeSeriesAnalysis June–July,2012 6/50
The first-order autoregressive process, AR(1)
(cid:3) The AR(1) process can be written using the notation of the lag operator, B,
defined by
Bz =z . (34)
t t−1
Letting z = z −µ and since Bz =z we have:
(cid:101)t t (cid:101)t (cid:101)t−1
(1−φB)z =a . (35)
(cid:101)t t
(cid:3) This condition indicates that a series follows an AR(1) process if on applying
the operator (1−φB) a white noise process is obtained.
(cid:3) The operator (1−φB) can be interpreted as a filter that when applied to the
series converts it into a series with no information, a white noise process.
AlonsoandGarc´ıa-Martos (UC3M-UPM) TimeSeriesAnalysis June–July,2012 7/50
The first-order autoregressive process, AR(1)
(cid:3) If we consider the operator as an equation, in B the coefficient φ is called the
factor of the equation.
(cid:3) The stationarity condition is that this factor be less than the unit in absolute
value.
(cid:3) Alternatively, we can talk about the root of the equation of the operator, which
is obtained by making the operator equal to zero and solving the equation with B
as an unknown;
1−φB =0
which yields B =1/φ.
(cid:3) The condition of stationarity is then that the root of the operator be greater
than one in absolute value.
AlonsoandGarc´ıa-Martos (UC3M-UPM) TimeSeriesAnalysis June–July,2012 8/50
The first-order autoregressive process, AR(1)
Expectation
(cid:3) Taking expectations in (33) assuming |φ|<1, such that E[z ]=E[z ]=µ,
t t−1
we obtain
µ=c +φµ
Then, the expectation (or mean) is
c
µ= (36)
1−φ
Replacing c in (33) with µ(1−φ), the process can be written in deviations to the
mean:
z −µ=φ(z −µ)+a
t t−1 t
and letting z =z −µ,
(cid:101)t t
z =φz +a (37)
(cid:101)t (cid:101)t−1 t
which is the most often used equation of the AR(1).
AlonsoandGarc´ıa-Martos (UC3M-UPM) TimeSeriesAnalysis June–July,2012 9/50
The first-order autoregressive process, AR(1)
Variance
(cid:3) The variance of the process is obtained by squaring the expression (37) and
taking expectations, which gives us:
E(z2)=φ2E(z2 )+2φE(z a )+E(a2).
(cid:101)t (cid:101)t−1 (cid:101)t−1 t t
We let σ2 be the variance of the stationary process. The second term of this
z
expression is zero, since as z and a are independent and both variables have
(cid:101)t−1 t
null expectation. The third is the variance of the innovation, σ2, and we conclude
that:
σ2 =φ2σ2+σ2,
z z
from which we find that the variance of the process is:
σ2
σ2 = . (38)
z 1−φ2
Note that in this equation the condition |φ|<1 appears, so that σ2 is finite and
z
positive.
AlonsoandGarc´ıa-Martos (UC3M-UPM) TimeSeriesAnalysis June–July,2012 10/50