Estimation of Stochastic Processes with Stationary Increments and Cointegrated Sequences

299 Pages·2019·3.422 MB·English

by  Maksym Luz, Mikhail Moklyachuk

Series Editor
Nikolaos Limnios
Estimation of Stochastic
Processes with Stationary
Increments and Cointegrated
Sequences

Maksym Luz
Mikhail Moklyachuk
First published 2019 in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc.
Apart from any fair dealing for the purposes of research or private study, or criticism or review, as
permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced,
stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers,
or in the case of reprographic reproduction in accordance with the terms and licenses issued by the
CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the
undermentioned address:
ISTE Ltd John Wiley & Sons, Inc.
27-37 St George’s Road 111 River Street
London SW19 4EU Hoboken, NJ 07030
UK USA
www.iste.co.uk www.wiley.com

© ISTE Ltd 2019
The rights of Maksym Luz and Mikhail Moklyachuk to be identified as the authors of this work have
been asserted by them in accordance with the Copyright, Designs and Patents Act 1988.
Library of Congress Control Number: 2019943878

British Library Cataloguing-in-Publication Data
A CIP record for this book is available from the British Library
ISBN 978-1-78630-503-9
Contents
Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Chapter 1. Stationary Increments of Discrete Time Stochastic
Processes: SpectralRepresentation . . . . . . . . . . . . . . . . . . . . 1
Chapter 2. Extrapolation Problem for Stochastic Sequences with
StationarynthIncrements . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1.Theclassicalmethodofextrapolation . . . . . . . . . . . . . . . . . . . 9
2.2.Minimax(robust)methodofextrapolation . . . . . . . . . . . . . . . . 21
2.3.LeastfavorablespectraldensityintheclassD0 . . . . . . . . . . . . . . 24
f
2.4.Least favorable spectral densities which admit factorization in the
classD0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
f
2.5.LeastfavorablespectraldensityintheclassDu . . . . . . . . . . . . . . 29
v
2.6.Least favorable spectral density which admits factorization in the
classDu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
v
Chapter 3. Interpolation Problem for Stochastic Sequences with
StationarynthIncrements . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.1.Theclassicalmethodofinterpolation . . . . . . . . . . . . . . . . . . . 31
3.2.Minimaxmethodofinterpolation . . . . . . . . . . . . . . . . . . . . . . 41
3.3.LeastfavorablespectraldensitiesintheclassD− . . . . . . . . . . . . 43
0,n
3.4.LeastfavorablespectraldensitiesintheclassD− . . . . . . . . . . . 47
M,n
vi EstimationofStochasticProcesses
Chapter 4. Extrapolation Problem for Stochastic Sequences
with Stationary nth Increments Based on Observations with
StationaryNoise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.1.Theclassicalmethodofextrapolationwithnoise . . . . . . . . . . . . . 53
4.2.Extrapolationofcointegratedstochasticsequences . . . . . . . . . . . . 71
4.3.Minimax(robust)methodofextrapolation. . . . . . . . . . . . . . . . . 75
4.4.LeastfavorablespectraldensitiesintheclassD0 ×D0 . . . . . . . . . 80
f g
4.5.Least favorable spectral densities which admit factorization in the
classD0 ×D0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
f g
4.6.LeastfavorablespectraldensitiesintheclassDu×D . . . . . . . . . 84
v ε
4.7.Least favorable spectral densities which admit factorization in the
classDu×D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
v ε
Chapter 5. Interpolation Problem for Stochastic Sequences
with Stationary nth Increments Based on Observations with
StationaryNoise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.1.Theclassicalmethodofinterpolationwithnoise . . . . . . . . . . . . . 89
5.2.Interpolationofcointegratedstochasticsequences . . . . . . . . . . . . 96
5.3.Minimax(robust)methodofinterpolation . . . . . . . . . . . . . . . . . 97
5.4.LeastfavorablespectraldensitiesintheclassD− ×D− . . . . . . . 100
0,f 0,g
5.5.LeastfavorablespectraldensitiesintheclassD ×D . . . . . . . 103
2ε1 1ε2
Chapter 6. Filtering Problem of Stochastic Sequences with
StationarynthIncrementsBasedonObservationswithStationary
Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
6.1.Theclassicalmethodoffiltering . . . . . . . . . . . . . . . . . . . . . . 107
6.2.Filteringproblemforcointegratedstochasticsequences . . . . . . . . . 119
6.3.Minimax(robust)methodoffiltering. . . . . . . . . . . . . . . . . . . . 124
6.4.LeastfavorablespectraldensitiesintheclassD0 ×D0 . . . . . . . . . 129
f g
6.5.Least favorable spectral densities which admit factorization in the
classD0 ×D0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
f g
6.6.LeastfavorablespectraldensitiesintheclassDu×D . . . . . . . . . 134
v ε
6.7.Least favorable spectral densities which admit factorization in the
classDu×D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
v ε
Chapter 7. Interpolation Problem for Stochastic Sequences with
StationarynthIncrementsObservedwithNon-stationaryNoise . . 139
7.1.Theclassicalinterpolationprobleminthecaseofnon-stationarynoise 140
7.2.Minimax(robust)methodofinterpolation . . . . . . . . . . . . . . . . . 148
7.3.LeastfavorablespectraldensitiesintheclassD− ×D− . . . . . . . 150
0,µ 0,µ
7.4.LeastfavorablespectraldensitiesintheclassD− ×D− . . . . . . 153
M,µ M,µ
Contents vii
Chapter 8. Filtering Problem for Stochastic Sequences with
StationarynthIncrementsObservedwithNon-stationaryNoise . . 155
8.1.Theclassicalfilteringprobleminthecaseofnon-stationarynoise . . . 156
8.2.Minimaxfilteringbasedonobservationswithnon-stationarynoise . . . 170
8.3.LeastfavorablespectraldensitiesintheclassD0 ×D0 . . . . . . . . . 174
f g
8.4.Least favorable spectral densities which admit factorizations in the
classD0 ×D0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
f g
8.5.LeastfavorablespectraldensitiesintheclassDv ×D . . . . . . . . . 177
u ε
8.6.Least favorable spectral densities which admit factorizations in the
classDu×D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
v ε
Chapter 9. Stationary Increments of Continuous Time Stochastic
Processes: SpectralRepresentation . . . . . . . . . . . . . . . . . . . . 181
Chapter 10. Extrapolation Problem for Stochastic Processes with
StationarynthIncrements . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
10.1.Hilbertspaceprojectionmethodofextrapolation . . . . . . . . . . . . 187
10.2.Minimax(robust)methodextrapolation . . . . . . . . . . . . . . . . . 205
10.3.LeastfavorablespectraldensitiesintheclassD0 ×D0 . . . . . . . . . 208
f g
10.4.LeastfavorablespectraldensityintheclassD0 . . . . . . . . . . . . . 210
f
10.5.Least favorable spectral density which admits factorization in the
classD0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
f
10.6.LeastfavorablespectraldensitiesintheclassDu×D . . . . . . . . . 213
v ε
10.7.Least favorable spectral densities which allow factorization in the
classD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
δ
Chapter 11. Interpolation Problem for Stochastic Processes with
StationarynthIncrements . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
11.1.Hilbertspaceprojectionmethodofinterpolation . . . . . . . . . . . . 217
11.2.Minimax(robust)methodofinterpolation . . . . . . . . . . . . . . . . 226
11.3.LeastfavorablespectraldensitiesintheclassD0 ×D0 . . . . . . . . . 229
f g
11.4.LeastfavorablespectraldensityintheclassD0 . . . . . . . . . . . . . 230
f
11.5.LeastfavorablespectraldensitiesintheclassD0 ×D0 . . . . . . 231
1/f 1/g
11.6.LeastfavorablespectraldensityintheclassD0 . . . . . . . . . . . . 233
1/f
11.7.LeastfavorablespectraldensitiesintheclassDu×D . . . . . . . . . 234
v ε
11.8.LeastfavorablespectraldensityintheclassDu . . . . . . . . . . . . . 235
v
11.9.LeastfavorablespectraldensityintheclassD . . . . . . . . . . . . . 236
2ε
Chapter 12. Filtering Problem for Stochastic Processes with
StationarynthIncrements . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
12.1.Hilbertspaceprojectionmethodoffiltering . . . . . . . . . . . . . . . 239
viii EstimationofStochasticProcesses
12.2.Minimax(robust)methodoffiltering . . . . . . . . . . . . . . . . . . . 246
12.3.LeastfavorablespectraldensitiesintheclassD0 ×D0 . . . . . . . . . 248
f g
12.4.LeastfavorablespectraldensitiesintheclassDu×D . . . . . . . . . 250
v ε
ProblemstoSolve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
Notations
N thesetofnaturalnumbers
Z thesetofintegernumbers
R thesetofrealnumbers
C( ) thesetofcomplexnum(b)ers
n Binomialcoefficient, n = n!
l l l!(n−l)!
[a]′ theleastintegernumberamongthenumbersthataregreaterthan
orequaltoa
(Ω,F,P) probabilityspace
Eξ mathematicalexpectationofarandomvariableξ
⟨· ; ·⟩ innerproduct
L (Ω,F,P) the Hilbert space of random variables with zero first moment,
2
finitesecondmomentandtheinnerproduct⟨ζ;η⟩=Eζη
L2(f) the Hilbert space o∫f the integrable functions with the inner
product⟨x ;x ⟩= π x (λ)x (λ)f(λ)dλ
1 2 −π 1 2
B thebackwardshiftoperatorwiththestepµ,B ξ(t)=ξ(t−µ)
µ µ
∆ξm incrementofthesequenceξm,∆ξm =ξm−ξm−1
ξ(n)(m,µ) stochasticnthincrementsequence(process)withastepµ
c(n)(µ) ameanvalueofthenthincrementsequence(process)
D(n)(m;µ ,µ ) a structural function of a stationary nth increment sequence
1 2
(process)
f,g spectraldensitiesoftheincrementsequences(processes)
F,G spectralfunctionsoftheincrementsequences(processes)
Aξ,A ξ alinearfunctionalofasequenceξ(m)
N
Aξ,A ξ alinearfunctionalofaprocessξ(t)
T
b
Aξ anestimateofafunctionalAξ
d′,F′ transposeofavectord′oralinearoperatorF
(d) kthelementofavectord
k
x EstimationofStochasticProcesses
[f(λ)] a positive part of the function: [f(λ)] = f(λ) if f(λ) ≥ 0,
+ +
[f(λ)] =0iff(λ)<0
+
h aspectralcharacteristicofanestimate
b b
∆(f;Aξ) ameansquareerroroftheestimateAξ
D asetofadmissiblespectraldensities
δ(·|D) theindicatorfunctionofasetD
∂∆D(f0) thesubdifferentialofafunctional∆D(f)atpointf =f0
It≥s anindicatorfunction: It≥s =1ift≥s,It≥s =0ift<s
I anidenticaloperator
F∗ anadjointoperatortoalinearoperatorF
z∗(λ),ε∗(t) an inverse Fourier transformation of a function z(λ) or of a
stochasticprocessε(t)
δ Kroneckerdelta: δ =1ifk =j,δ =0ifk ̸=j
k,j k,j k,j
dom∆ forafunctional∆,dom∆:={f ∈X :∆(f)<∞}
Introduction
Theproblemofestimationoftheunobservedvaluesofstochasticprocessesplaysa
keyroleinvariousareasofappliedresearch.Theproblemofforecastingfuturevalues
of economic and physical processes, and the problem of restoring lost information,
cleaningsignalsorotherdataobservationsfromnoisearemagnifiedinaninformation-
laden world. Therefore, the development of estimation methods is one of the main
tasksofthemoderntheoryofstochasticprocesses.
Methodsofstochasticprocessesestimationdependontwomainfactors.Thefirst
factor is the construction of a model of the process being investigated. The classical
problems of filtering (smoothing), interpolation (estimation of missing values),
andextrapolation(forecasting)ofstochasticprocessesareinvestigatedforstationary
processesandsequenceswhichareconsideredasmodelsoftheprocesses.Thesecond
factor,whichplaysamajorroleinchoosingthemethodofestimation,istheavailable
information about the structure of the process under consideration. In particular, in
thecaseofstationaryprocesses,theoreticalresultsareobtainedundertheassumption
thatresearchersknewthecorrelationorspectralfunctionofsuchprocesses.However,
practical applications of the theory required further development of the proposed
estimationmethods,sinceresearchersusuallyhaveincompleteinformationaboutthe
process characteristics and can only get, for example, estimates of some parameters
ofthespectralcharacteristicsoftheprocessornon-parametricestimateofthespectral
functionoftheprocess.Therefore,indevelopingthemethodsofestimationofvalues
of stochastic processes, one should take into account the theoretical model of the
process, as well as the fact that complete information about model of the process
underinvestigationmaynotbeavailable.
Nowadays, the theory of estimation of unknown values of stationary stochastic
processes is developed sufficiently. A key feature of these processes is that their
spectralcharacteristicsdonotchangeovertime,whichallowsustoconstructestimates
from the observed data. For stationary stochastic processes with continuous and
xii EstimationofStochasticProcesses
discretetime,theseproblemsweresetandsolvedbyN.WienerandA.N.Kolmogorov.
In particular, the work by A. N. Kolmogorov [KOL41b] provides an overview of
the spectral theory of stationary sequences. He considered stationary and related
stochasticsequences.Themainpropertiesofsuchsequenceswereobtainedbasedon
the H. Crame´r [CRA40] and M. Stone [STO32] results. The stationary sequences
were described by the corresponding unitary operator U acting on a stationary
sequence x(t), t ∈ Z, by the rule Ux(t) = x(t + 1). The key properties such as
spectralrepresentationofthecovariancefunction,propertiesofthespectralfunction,
characterization of subordinated stationary sequences, decomposition of stationary
sequences on orthogonal components, and a moving-average representation were
obtained by the author as a consequence of the results of the spectral theory of
unitary operators. Based on the results by H. Wold [WOL54], A. N. Kolmogorov
proposed a decomposition of stationary sequences into a sum of regular and
singular stationary sequences. He also derived a number of properties of stationary
sequences and received characterization of regular stationary sequences, introduced
t∫heconceptofminimalityandwasthefirstonewhoprovedtheminimalitycondition
π f−1(λ)dλ<∞forthespectraldensityf(λ).Theseresultswereusedforsolving
−π
theextrapolationandinterpolationproblemsforstationarysequencesx(t),t ∈ Z.In
particular,A.N.Kolmogorov[KOL41a]consideredtheproblemofestimationofthe
unknownvaluex(t+m),m≥0,approximatingitbyalinearcombination
L=a x(t−1)+a x(t−2)+...+a x(t−n).
1 2 n
As a measure of optimality, the value σ2(m,n) = E|x(t+m)−L|2 was chosen.
E
In the cited article, the author solved the problem of finding the value of σ2(m) =
E
limn→∞σE2(m,n).Thesecondproblemintroducedinthearticlewastheinterpolation
problem, namely, estimating the unknown value x(t) by approximating it by the
functional
Q=a1x(t+1)+...+anx(t+n)+a−1x(t−1)+...+a−nx(t−n).
For this problem, the author considered the value of σ2(n) = E|x(t) − Q|2 as a
I
measure of accuracy and found the limit σI2 = limn→∞σI2(n). It should be noted
thatA.N.Kolmogorov[KOL41a]foundvaluesofthemeansquareerrorsσ2(m)and
I
σ2.Formulaswhichallowcalculatingestimatesoftheunknownvaluesofstationary
I
processescanbefoundintheworksbyH.Wold[WOL48],A.M.Yaglom[YAG55,
YAG87a,YAG87b],andY.Rozanov[ROZ58].
Spectral theory for the continuous time stationary processes was developed
by A. Khinchin [KHI34]. Extrapolation, interpolation, and filtering problems for
continuoustimestationaryprocesseswereinvestigatedbyN.Wiener[WIE66].
Inthesamewayasthetheoryofone-dimensionalstationaryprocesses,thetheory
ofvector-valuedstationaryprocesseswasdeveloped.Oneofthefirstworksdedicated