ARMA Models for Stationary Time Series

21 Pages·2009·4.07 MB·English

by  Robert Stine

ARMA Models for
Stationary Time Series
INSR 260, Spring 2009
Bob Stine
1
Over view
Stationarity
True Sample
Correlation functions
Autocorrelation function (TAC and SAC, ACF)
Partial autocorrelation function (TPAC and SPAC, PACF)
Models
Autoregression
Moving average
Combination
Examples
GDP, paper towel sales
Simulation of processes
2
Stationarity
Stationarity is a key assumption in time series.
Stationarity = properties of the time series do not
depend on when time starts.
Second-order stationary is most common form assumed in practice
Constant mean and variance
! ! E(Y ) = μ !! Var(Y ) = σ 2
t t y
Autocorrelation only depends only on separation
! ! Corr(Y ,Y ) = ρ !!
t t-k k
Why does stationarity matter?
Without stationarity, we’ve got nothing to average.
Two approaches to obtaining stationary data
Differencing! ! ! ! Model changes Y - Y
t t-1
Regression
3
Differencing
Raw data has possible trends, but differencing
removes most of this appearance
Visual procedure
Heuristic
Further differencing is also possible and sometimes needed
Later modeling allows more precise analysis
3
20
2
15
1
0)
0
0 e
010 g 0
1 n
x a
s ( Ch
oll -1
R 5
-2
0
first differences
-3
0 10 20 30 40 50 60 70 80 90 100 110 0 10 20 30 40 50 60 70 80 90 100 110 120
Week
Week
Table 9-1 Paper towels
4
GDP Example
Predict annual US GDP! ! ! (real dollars)
Data span 1929-2008
Use data after WWII (1946-2008)
Remarkably linear on log scale
R2 ≈ 99%
About 7% annual rate of growth (includes inflation)
20000
10000
6000
Parameter Estimates
4000
Term Estimate Std Error t Ratio Prob>|t|
P
D 2000 Intercept -133.3041 1.629637 -81.80 <.0001*
G Year 0.0712604 0.000824 86.45 <.0001*
1000
l
a
u 600
n
400
n
A
200
100 Concerns?
60
40
1930 1940 1950 1960 1970 1980 1990 2000
Year
5
GDP Residuals
Pattern in residuals from linear regression
Residuals suggest stationary time series
Meandering pattern implies dependence
Oscillates around fit of model on log scale
Regression over-predicts GDP since 2000
Autocorrelation between adjacent residuals
! ! ! ! corr(e ,e )≈0.94
t t-1
Durbin-Watson
0.2
Durbin- Number
Watson of Obs. AutoCorrelation
P
D 0.0596695 63 0.9411
0.1
G

g
o
L
0
s
l
a
u
d
i Look back:
s-0.1
e
R
Pattern becomes evident
-0.2 in scatterplot of GDP on
time once you have seen
1950 1960 1970 1980 1990 2000 2010
it in the residual plot.
Year
6
Improved Model
Follow Durbin-Watson approach
Add lag of residuals back into equation Term Estimate Std Error t Ratio
Intercept -132.7244 0.403924 -328.6
Year 0.0709652 0.000204 347.43
Refit regression with lagged variable Lag Log Residuals GDP 1.0013018 0.031908 31.38
Multiple regression with lagged variable
Very high R2 (0.9995)
Durbin- Number
Watson of Obs. AutoCorrelation Prob<DW
But DW remains significant
1.5222692 62 0.2290 0.0143*
Does more dependence remain in the residuals?
0.05
g
g
a
o
L
L h 0
l
a t
i
u w
d
si P,
e D
R G
-0.05
-0.1
1940 1950 1960 1970 1980 1990 2000 2010
Year
7
Closer Look At DW Stat
Durbin-Watson statistic tests for a specific type
of dependence among residuals:
! ! ! first-order autocorrelation
Model: First-order autoregression !! AR(1)
Equation
! ! (y -μ) = φ(y -μ) + a ! ! ! ! μ= E(y )
t t-1 t t
Errors a have mean zero, equal variance, independent
t
! ! E(a ) = 0, !! Var(a ) = σ2, !! Corr(a ,a )=0, s≠t
t t t s
Questions
How do we identify whether the dependence among the
errors in our model is AR(1) or some other type?
What other types are there?
8
Recognizing AR(1) Dependence
A first-order autoregression is a geometrically
weighted sum of past error terms.
Assume |φ| < 1
Back-substitute, assuming μ = 0!! ! (e.g. residuals)
! ! y != φy + a
t t-1 t
! ! ! = φ(φy + a ) + a
t-2 t-1 t
! ! ! = φ(φ(φy + a )+ a ) + a
t-3 t-2 t-1 t
! ! ! = a + φa + φ2a + φ3y
t t-1 t-2 t-3
! ! ! = a + φa + φ2a + … + φma + …!
t t-1 t-2 t-m
Autocorrelation decays
First two
! ! Corr(y ,y ) = Corr(φy + a , y ) = φ
t t-1 t-1 t t-1
! ! Corr(y ,y ) = Corr(φ2y + φa + a , y ) = φ2
t t-2 t-2 t-1 t t-1
In general
! ! Corr(y ,y ) = φk, k = 1,2,3...
t t-k
9
Autocorrelation Function
Identification procedure
Autocorrelations decay geometrically if process is AR(1)
Plot the autocorrelations; decide if decay looks geometric.
Notation
! ! TAC(k) = ρ = Correlation(Y , Y )!! ! [a.k.a., ACF]
k t t-k
Complications
We observe sample correlations (SAC r ), not true values
k
Outliers, nonlinearity can make a mess of things
How close to geometric is close enough?
Calculation
JMP plots the autocorrelation function for you without
having to form all of the lags (beware outliers!)
Analyze > Modeling > Time Series
10