Multidimensional Stationary Time Series: Dimension Reduction and Prediction

296 Pages·2021·9.552 MB·English

by  Marianna Bolla, Tamás Szabados

Multidimensional
Stationary Time Series

Multidimensional
Stationary Time Series
Dimension Reduction and
Prediction
Marianna Bolla
Tamás Szabados
First edition published 2021
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742
and by CRC Press
2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
© 2021 Marianna Bolla and Tamás Szabados
CRC Press is an imprint of Taylor & Francis Group, LLC
The right of Marianna Bolla and Tamás Szabados to be identified as authors of this work has been as-
serted by them in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988.
Reasonable efforts have been made to publish reliable data and information, but the author and pub-
lisher cannot assume responsibility for the validity of all materials or the consequences of their use.
The authors and publishers have attempted to trace the copyright holders of all material reproduced
in this publication and apologize to copyright holders if permission to publish in this form has not
been obtained. If any copyright material has not been acknowledged please write and let us know so
we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information stor-
age or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, access www.copyright.
com or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA
01923, 978-750-8400. For works that are not available on CCC please contact mpkbookspermis-
[email protected]
Trademark notice: Product or corporate names may be trademarks or registered trademarks and are
used only for identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
ISBN: 9780367569327 (hbk)
ISBN: 9781003107293 (ebk)
Typeset in CMR10 font
by KnowledgeWorks Global Ltd.
To our families.

Contents
Foreword xi
Preface xiii
List of Figures xvii
Symbols xix
1 Harmonic analysis of stationary time series 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Covariance function and spectral representation . . . . . . . 1
1.3 Spectralrepresentationofmultidimensionalstationarytimese-
ries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.4 Constructions of stationary time series . . . . . . . . . . . . 18
1.4.1 Construction 1 . . . . . . . . . . . . . . . . . . . . . . 18
1.4.2 Construction 2 . . . . . . . . . . . . . . . . . . . . . . 20
1.4.3 Construction 3 . . . . . . . . . . . . . . . . . . . . . . 22
1.4.4 Construction 4 . . . . . . . . . . . . . . . . . . . . . . 23
1.4.4.1 Discrete Fourier Transform . . . . . . . . . . 23
1.4.4.2 The construction . . . . . . . . . . . . . . . . 25
1.5 Estimating parameters of stationary time series . . . . . . . 26
1.5.1 Estimation of the mean . . . . . . . . . . . . . . . . . 26
1.5.2 Estimation of the covariances . . . . . . . . . . . . . . 30
1.5.3 Periodograms . . . . . . . . . . . . . . . . . . . . . . . 33
1.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2 ARMA, regular, and singular time series in 1D 39
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.2 Time invariant linear filtering . . . . . . . . . . . . . . . . . 40
2.3 Moving Average processes . . . . . . . . . . . . . . . . . . . 42
2.4 Autoregressive processes . . . . . . . . . . . . . . . . . . . . 46
2.5 Autoregressive moving average processes . . . . . . . . . . . 53
2.6 Wold decomposition in 1D . . . . . . . . . . . . . . . . . . . 58
2.7 Spectral form of the Wold decomposition . . . . . . . . . . . 60
2.8 Factorization of rational and smooth densities . . . . . . . . 65
2.8.1 Rational spectral density . . . . . . . . . . . . . . . . 65
2.8.2 Smooth spectral density . . . . . . . . . . . . . . . . . 66
vii
viii Contents
2.9 Classification of stationary time series in 1D . . . . . . . . . 67
2.10 Examples for singular time series . . . . . . . . . . . . . . . . 75
2.10.1 Type (0) singular time series . . . . . . . . . . . . . . 75
2.10.2 Type (1) singular time series . . . . . . . . . . . . . . 78
2.10.3 Type (2) singular time series . . . . . . . . . . . . . . 78
2.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
3 Linear system theory, state space models 87
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.2 Restricted input/output map . . . . . . . . . . . . . . . . . . 87
3.3 Reachability and observability . . . . . . . . . . . . . . . . . 89
3.4 Power series and extended input/output maps . . . . . . . . 90
3.5 Realizations . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.6 Stochastic linear systems . . . . . . . . . . . . . . . . . . . . 103
3.6.1 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.6.2 Prediction, miniphase condition, and covariance. . . . 105
3.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
4 Multidimensional time series 113
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
4.2 Linear transformations, subordinated processes . . . . . . . . 113
4.3 Stationary time series of constant rank . . . . . . . . . . . . 116
4.4 Multidimensional Wold decomposition . . . . . . . . . . . . . 120
4.4.1 Decomposition with an orthonormal process . . . . . . 120
4.4.2 Decomposition with innovations . . . . . . . . . . . . 122
4.5 Regular and singular time series . . . . . . . . . . . . . . . . 124
4.5.1 Full rank processes . . . . . . . . . . . . . . . . . . . . 126
4.5.2 Generic regular processes . . . . . . . . . . . . . . . . 135
4.5.3 Classification of non-regular multidimensional time se-
ries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
4.6 Low rank approximation . . . . . . . . . . . . . . . . . . . . 141
4.6.1 Approximation of time series of constant rank . . . . . 142
4.6.2 Approximation of regular time series . . . . . . . . . . 146
4.7 Rational spectral densities . . . . . . . . . . . . . . . . . . . 147
4.7.1 Smith–McMillan form . . . . . . . . . . . . . . . . . . 148
4.7.2 Spectral factors of a rational spectral density matrix . 150
4.8 Multidimensional ARMA (VARMA) processes . . . . . . . . 151
4.8.1 Equivalence of different approaches . . . . . . . . . . . 151
4.8.2 Yule–Walker equations . . . . . . . . . . . . . . . . . . 158
4.8.3 Prediction, miniphase condition, and approximation by
VMA processes . . . . . . . . . . . . . . . . . . . . . . 161
4.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Contents ix
5 Dimension reduction and prediction in the time and fre-
quency domain 169
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
5.2 1D prediction in the time domain . . . . . . . . . . . . . . . 170
5.2.1 One-step ahead prediction based on finitely many past
values . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
5.2.2 Innovations . . . . . . . . . . . . . . . . . . . . . . . . 173
5.2.3 Prediction based on the infinite past . . . . . . . . . . 175
5.3 Multidimensional prediction . . . . . . . . . . . . . . . . . . 178
5.3.1 One-step ahead prediction based on finitely many past
values . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
5.3.2 Multidimensional innovations . . . . . . . . . . . . . . 180
5.4 Spectra of spectra . . . . . . . . . . . . . . . . . . . . . . . . 183
5.4.1 Bounds for the eigenvalues of C . . . . . . . . . . . . 189
n
5.4.2 Principal component transformation as discrete Fourier
transformation . . . . . . . . . . . . . . . . . . . . . . 190
5.5 K´alm´an’s filtering . . . . . . . . . . . . . . . . . . . . . . . . 191
5.6 Dynamic principal component and factor analysis . . . . . . 199
5.6.1 Time domain approach via innovations. . . . . . . . . 199
5.6.2 Frequency domain approach . . . . . . . . . . . . . . . 201
5.6.3 Best low-rank approximation in the frequency domain,
and low-dimensional approximation in the time domain 202
5.6.4 Dynamic factor analysis . . . . . . . . . . . . . . . . . 204
5.6.5 General Dynamic Factor Model . . . . . . . . . . . . . 207
5.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
A Tools from complex analysis 215
A.1 Holomorphic (or analytic) functions . . . . . . . . . . . . . . 215
A.2 Harmonic functions . . . . . . . . . . . . . . . . . . . . . . . 219
A.3 Hardy spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
A.3.1 First approach . . . . . . . . . . . . . . . . . . . . . . 222
A.3.2 Second approach . . . . . . . . . . . . . . . . . . . . . 225
B Matrix decompositions and special matrices 227
C Best prediction in Hilbert spaces 241
D Tools from algebra 249
Bibliography 265
Index 269