Nonlinear Time Series: Theory, Methods and Applications with R Examples

554 Pages·2014·9.465 MB·English

by  Randal Douc, Eric Moulines, David Stoffer

Texts in Statistical Science
r i[N
inear
I
eries
im:e
Theory, Methods, and
Applications with R Examples
o. h Al
A
i
T III
111 0 J
'k
oa
T
zo no goo 2g ao so 8o tog
time time
0
T
00 0%
O O
O- i
Oops 4
O
4- .11
-20 -10 0 10' =Aol =1g. O= no.
XI Awl
R an, dial D o u c
Eric Moulines-
David S. Stoffe r
CRC Press
Taylor & Francis Croup
Aa CHAPMA'N' & HALL
Nonlinear
Time Series
Theory, Methods, and
Applications with R Examples
Randal Douc
Eric Moulines
David Stoffer

CHAPMAN & HALL/CRC
Texts in Statistical Science Series
Series Editors
Francesca Dominici, Harvard School of Public Health, USA
Julian J. Faraway, University of Bath, UK
Martin Tanner, Northwestern University, USA
Jim Zidek, University of British Columbia, Canada
Statistical Theory: A Concise Introduction Introduction to Statistical Methods for
F. Abramovich and Y. Ritov Clinical Trials
T.D. Cook and D.L. DeMets
Practical Multivariate Analysis, Fifth Edition
A. Afifi, S. May, and V.A. Clark Applied Statistics: Principles and Examples
Practical Statistics for Medical Research D.R. Cox and E.J. Snell
D.G. Altman Multivariate Survival Analysis and Competing
Interpreting Data: A First Course Risks
in Statistics M. Crowder
A.J.B. Anderson Statistical Analysis of Reliability Data
Introduction to Probability with R M.J. Crowder, A.C. Kimber,
K. Baclawski T.J. Sweeting, and R.L. Smith
Linear Algebra and Matrix Analysis for An Introduction to Generalized
Statistics Linear Models, Third Edition
S. Banerjee and A. Roy A.J. Dobson and A.G. Barnett
Statistical Methods for SPC and TQM Nonlinear Time Series: Theory, Methods, and
D. Bissell Applications with R Examples
R. Douc, E. Moulines, and D. Stoffer
Bayesian Methods for Data Analysis,
Third Edition Introduction to Optimization Methods and
B.P. Carlin and T.A. Louis Their Applications in Statistics
B.S. Everitt
Second Edition
R. Caulcutt Extending the Linear Model with R:
Generalized Linear, Mixed Effects and
The Analysis of Time Series: An Introduction,
Nonparametric Regression Models
Sixth Edition
J.J. Faraway
C. Chatfield
A Course in Large Sample Theory
Introduction to Multivariate Analysis
T.S. Ferguson
C. Chatfield and A.J. Collins
Multivariate Statistics: A Practical Approach
Problem Solving: A Statistician’s Guide,
B. Flury and H. Riedwyl
Second Edition
C. Chatfield Readings in Decision Analysis
S. French
Statistics for Technology: A Course in Applied
Statistics, Third Edition Markov Chain Monte Carlo:
C. Chatfield Stochastic Simulation for Bayesian Inference,
Second Edition
Bayesian Ideas and Data Analysis: An
D. Gamerman and H.F. Lopes
Introduction for Scientists and Statisticians
Bayesian Data Analysis, Third Edition
R. Christensen, W. Johnson, A. Branscum,
A. Gelman, J.B. Carlin, H.S. Stern, D.B. Dunson,
and T.E. Hanson
A. Vehtari, and D.B. Rubin
Modelling Binary Data, Second Edition
Multivariate Analysis of Variance and
D. Collett
Repeated Measures: A Practical Approach for
Modelling Survival Data in Medical Research,
Behavioural Scientists
Second Edition
D.J. Hand and C.C. Taylor
D. Collett
Practical Data Analysis for Designed Practical The BUGS Book: A Practical Introduction to
Longitudinal Data Analysis Bayesian Analysis
D.J. Hand and M. Crowder D. Lunn, C. Jackson, N. Best, A. Thomas, and
D. Spiegelhalter
Logistic Regression Models
J.M. Hilbe Introduction to General and Generalized
Linear Models
Richly Parameterized Linear Models:
H. Madsen and P. Thyregod
Additive, Time Series, and Spatial Models
Using Random Effects Time Series Analysis
J.S. Hodges H. Madsen
Statistics for Epidemiology Pólya Urn Models
N.P. Jewell H. Mahmoud
Stochastic Processes: An Introduction, Randomization, Bootstrap and Monte Carlo
Second Edition Methods in Biology, Third Edition
P.W. Jones and P. Smith B.F.J. Manly
The Theory of Linear Models Introduction to Randomized Controlled
B. Jørgensen Clinical Trials, Second Edition
J.N.S. Matthews
Principles of Uncertainty
J.B. Kadane Statistical Methods in Agriculture and
Experimental Biology, Second Edition
Graphics for Statistics and Data Analysis with R
R. Mead, R.N. Curnow, and A.M. Hasted
K.J. Keen
Statistics in Engineering: A Practical Approach
Mathematical Statistics
A.V. Metcalfe
K. Knight
Beyond ANOVA: Basics of Applied Statistics
Nonparametric Methods in Statistics with SAS
Applications R.G. Miller, Jr.
O. Korosteleva A Primer on Linear Models
Modeling and Analysis of Stochastic Systems, J.F. Monahan
Second Edition Applied Stochastic Modelling, Second Edition
V.G. Kulkarni B.J.T. Morgan
Exercises and Solutions in Biostatistical Theory Elements of Simulation
L.L. Kupper, B.H. Neelon, and S.M. O’Brien B.J.T. Morgan
Exercises and Solutions in Statistical Theory Probability: Methods and Measurement
L.L. Kupper, B.H. Neelon, and S.M. O’Brien A. O’Hagan
Design and Analysis of Experiments with SAS Introduction to Statistical Limit Theory
J. Lawson A.M. Polansky
A Course in Categorical Data Analysis Applied Bayesian Forecasting and Time Series
T. Leonard Analysis
Statistics for Accountants A. Pole, M. West, and J. Harrison
S. Letchford Statistics in Research and Development,
Introduction to the Theory of Statistical Time Series: Modeling, Computation, and
Inference Inference
H. Liero and S. Zwanzig R. Prado and M. West
Statistical Theory, Fourth Edition Introduction to Statistical Process Control
B.W. Lindgren P. Qiu
Stationary Stochastic Processes: Theory and Sampling Methodologies with Applications
Applications P.S.R.S. Rao
G. Lindgren
Texts in Statistical Science
Nonlinear
Time Series
Theory, Methods, and
Applications with R Examples
Randal Douc
Telecom SudParis
Evry, France
Randal Douc
Eric Moulines
Eric Moulines
Telecom ParisTech
Paris, France
David Stoffer
David Stoffer
University of Pittsburgh
Pennsylvania, USA
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2014 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed on acid-free paper
Version Date: 20131111
International Standard Book Number-13: 978-1-4665-0225-3 (Hardback)
This book contains information obtained from authentic and highly regarded sources. Reasonable
efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and
publishers have attempted to trace the copyright holders of all material reproduced in this publication
and apologize to copyright holders if permission to publish in this form has not been obtained. If any
copyright material has not been acknowledged please write and let us know so we may rectify in any
future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information stor-
age or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copy-
right.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222
Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that pro-
vides licenses and registration for a variety of users. For organizations that have been granted a pho-
tocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are
used only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
Preface xiii
FrequentlyUsedNotation xvii
I Foundations 1
1 LinearModels 3
1.1 Stochasticprocesses 3
1.2 Thecovarianceworld 5
1.2.1 Second-orderstationaryprocesses 5
1.2.2 Spectralrepresentation 9
1.2.3 Wolddecomposition 13
1.3 Linearprocesses 15
1.3.1 WhatarelinearGaussianprocesses? 15
1.3.2 ARMAmodels 16
1.3.3 Prediction 19
1.3.4 Estimation 21
1.4 Themultivariatecases 25
1.4.1 Timedomain 25
1.4.2 Frequencydomain 27
1.5 Numericalexamples 28
Exercises 30
2 LinearGaussianStateSpaceModels 33
2.1 Modelbasics 33
2.2 Filtering,smoothing,andforecasting 36
2.3 Maximumlikelihoodestimation 42
2.3.1 Newton–Raphson 42
2.3.2 EMalgorithm 43
2.4 SmoothingsplinesandtheKalmansmoother 45
2.5 AsymptoticdistributionoftheMLE 47
2.6 Missingdatamodifications 49
2.7 Structuralcomponentmodels 50
2.8 State-spacemodelswithcorrelatederrors 53
2.8.1 ARMAXmodels 54
vii
viii CONTENTS
2.8.2 Regressionwithautocorrelatederrors 56
Exercises 56
3 BeyondLinearModels 61
3.1 Nonlinearnon-Gaussiandata 62
3.2 Volterraseriesexpansion 68
3.3 Cumulantsandhigher-orderspectra 69
3.4 Bilinearmodels 72
3.5 Conditionallyheteroscedasticmodels 73
3.6 ThresholdARMAmodels 77
3.7 Functionalautoregressivemodels 78
3.8 Linearprocesseswithinfinitevariance 79
3.9 Modelsforcounts 81
3.9.1 Integervaluedmodels 81
3.9.2 Generalizedlinearmodels 83
3.10 Numericalexamples 84
Exercises 89
4 StochasticRecurrenceEquations 91
4.1 TheScalarCase 93
4.1.1 Strictstationarity 93
4.1.2 Weakstationarity 98
4.1.3 GARCH(1,1) 102
4.2 TheVectorCase 107
4.2.1 Strictstationarity 109
4.2.2 Weakstationarity 111
4.2.3 GARCH(p,q) 114
4.3 Iteratedrandomfunction 118
4.3.1 Strictstationarity 118
4.3.2 Weakstationarity 121
Exercises 123
II MarkovianModels 131
5 MarkovModels:ConstructionandDefinitions 133
5.1 Markovchains:Past,future,andforgetfulness 133
5.2 Kernels 134
5.3 HomogeneousMarkovchain 136
5.4 Canonicalrepresentation 138
5.5 Invariantmeasures 139
5.6 Observation-drivenmodels 142
5.7 Iteratedrandomfunctions 143
5.8 MCMCmethods 152
5.8.1 Metropolis-Hastingsalgorithm 152
5.8.2 Gibbssampling 155
CONTENTS ix
Exercises 157
6 StabilityandConvergence 165
6.1 Uniformergodicity 166
6.1.1 Totalvariationdistance 166
6.1.2 Dobrushincoefficient 167
6.1.3 TheDoeblincondition 169
6.1.4 Examples 169
6.2 V-geometricergodicity 173
6.2.1 V-totalvariationdistance 173
6.2.2 V-Dobrushincoefficient 174
6.2.3 Driftandminorizationconditions 175
6.2.4 Examples 180
6.3 Someproofs 186
6.4 Endnotes 188
Exercises 189
7 SamplePathsandLimitTheorems 195
7.1 Lawoflargenumbers 196
7.1.1 Dynamicalsystemandergodicity 196
7.1.2 Markovchainergodicity 203
7.2 Centrallimittheorem 211
7.3 Deviationinequalitiesforadditivefunctionals 218
7.3.1 Rosenthaltypeinequality 218
7.3.2 Concentrationinequality 221
7.4 Someproofs 225
Exercises 231
8 InferenceforMarkovianModels 239
8.1 Likelihoodinference 239
8.2 ConsistencyandasymptoticnormalityoftheMLE 245
8.2.1 Consistency 245
8.2.2 Asymptoticnormality 247
8.3 Observation-drivenmodels 254
8.4 Bayesianinference 263
8.5 Someproofs 271
8.6 Endnotes 274
Exercises 275
III StateSpaceandHiddenMarkovModels 285
9 Non-GaussianandNonlinearStateSpaceModels 287
9.1 Definitionsandbasicproperties 287
9.1.1 Discrete-valuedstatespaceHMM 287
9.1.2 Continuous-valuedstate-spacemodels 295