Modeling of stationary periodic time series by ARMA representations

34 Pages·2016·2.31 MB·English

Modeling of Stationary Periodic Time Series by
ARMA Representations
AndersLindquistandGiorgioPicci
DedicatedtoBorisTeodorovichPolyak ontheoccasionofhis
80thbirthday
Abstract Thisisasurveyofsomerecentresultsontherationalcirculantcovariance
extension problem: Given a partial sequence .c ;c ;:::;c / of covariance lags
0 1 n
ck E y.t k/y.t/ emanatingfromastationaryperiodicprocess y.t/ withperiod
D f C g f g
2N > 2n,findallpossiblerationalspectralfunctionsof y.t/ ofdegreeatmost2n
f g
or,equivalently,allbilateralandunilateralARMAmodelsoforderatmostn,having
thispartialcovariancesequence.Eachrepresentationisobtainedasthesolutionofa
pairofdualconvexoptimizationproblems.Thistheoryisthenreformulatedinterms
ofcirculantmatricesandtheconnectionstoreciprocalprocessesandthecovariance
selectionproblemisexplained.Nextitisshownhowthetheorycanbeextendedto
themultivariatecase.Finally,anapplicationtoimageprocessingispresented.
Keywords Discretemomentproblem • Periodicprocesses • Circulantcovariance
extension • BilateralARMAmodels • Imageprocessing
1 Introduction
The rational covariance extension problem to determine a rational spectral den-
sity given a finite number of covariance lags has been studied in great detail
[2, 5–7, 9, 10, 17, 19, 20, 24, 35], and it can be formulated as a (truncated)
trigonometricmomentproblemwithadegreeconstraint.Amongotherthings,itis
the basic problem in partial stochastic realization theory [2] and certain Toeplitz
A.Lindquist(!)
ShanghaiJiaoTongUniversity,Shanghai,China
RoyalInstituteofTechnology,Stockholm,Sweden
e-mail:[email protected]
G.Picci
UniversityofPadova,Padova,Italy
e-mail:[email protected]
©SpringerInternationalPublishingSwitzerland2016 281
B.Goldengorin(ed.),OptimizationandItsApplicationsinControl
andDataSciences,SpringerOptimizationandItsApplications115,
DOI10.1007/978-3-319-42056-1_9
282 A.LindquistandG.Picci
matrix completion problems. In particular, it provides a parameterization of the
familyof(unilateral)autoregressivemoving-average(ARMA)modelsofstationary
stochasticprocesseswiththesamefinitesequenceofcovariancelags.Wealsorefer
thereadertotherecentmonograph[31],inwhichthisproblemisdiscussedinthe
contextofstochasticrealizationtheory.
Covarianceextensionforperiodicstochasticprocesses,ontheotherhand,leads
to matrix completion of Toeplitz matrices with circulant structure and to partial
stochasticrealizationsintheformofbilateralARMAmodels
n n
q y.t k/ p e.t k/
k k
! D !
k n k n
XD! XD!
for a stochastic processes y.t/ , where e.t/ is the corresponding conjugate
f g f g
process. This connects up to a rich realization theory for reciprocal processes
[26–29]. As we shall see there are also (forward and backward) unilateral ARMA
representationsforperiodicprocesses.
In[12]amaximum-entropyapproachtothiscirculantcovarianceextensionprob-
lemwaspresented,providingaprocedurefordeterminingtheuniquebilateralAR
modelmatchingthecovariancesequence.However,morerecentlyitwasdiscovered
that the circulant covariance extension problem can be recast in the context of the
optimization-based theory of moment problems with rational measures developed
in[1,3,4,6,8–10,21,22]allowingforacompleteparameterizationofallbilateral
ARMA realizations. This led to a complete theory for the scalar case [30], which
wasthenextendedtothemultivariablecasein[32].Alsosee[38]formodifications
ofthistheorytoskewperiodicprocessesand[37]forfastnumericalprocedures.
The AR theory of [12] has been successfully applied to image processing of
textures [13,36], and we anticipate an enhancement of such methods by allowing
formoregeneralARMArealizations.
Thepresentsurveypaperistoalargeextentbasedon[30,32]and[12].InSect.2
we begin by characterizing stationary periodic processes. In Sect.3 we formulate
the rational covariance extension problem for periodic processes as a moment
problemwithatomicmeasureandpresentthesolutioninthecontextoftheconvex
optimization approach of [1,3,4,6,8–10]. These results are then reformulated in
terms of circulant matrices in Sect.4 and interpreted in term of bilateral ARMA
modelsinSect.5andintermsofunilateralARMAmodelsinSect.6.InSect.7we
investigatetheconnectionstoreciprocalprocessesofordern[12]andthecovariance
selection problem of Dempster [15]. In Sect.8 we consider the situation when
bothpartialcovariancedataandlogarithmicmoment(cepstral)dataisavailable.To
simplifytheexpositionthetheoryhassofarbeendevelopedinthecontextofscalar
processes,butinSect.9weshowhowitcanbeextendedtothemultivariablecase.
Alloftheseresultsareillustratedbyexamplestakenfrom[30]and[32].Section10
isdevotedtoapplicationsinimageprocessing.
ModelingofStationaryPeriodicTimeSeriesbyARMARepresentations 283
2 PeriodicStationaryProcesses
Consider a zero-mean full-rank stationary process y.t/ , in general complex-
f g
valued,definedonafiniteintervalΠN 1; N!oftheintegerlineZandextended
! C
toallofZasaperiodicstationaryprocesswithperiod2N sothat
y.t 2kN/ y.t/ (1)
C D
almostsurely.Bystationaritythereisarepresentation
"
y.t/ eit#dy.#/; whereE dy2 dF.#/; (2)
D O fj Oj gD
Z "
!
(see,e.g.,[31,p.74]),andtherefore
"
ck E y.t k/y.t/ eik#dF.#/: (3)
WD f C gD
Z "
!
Also,inviewof(1),
"
eit# ei2N# 1 dy 0;
! O D
Z "
! ! "
andhence
"
ei2N# 1 2dF 0;
! D
Z "
! ˇ ˇ
ˇ ˇ
which shows that the support of dF must be contained in k"=N k N
f I D! C
1;:::;N . Consequently the spectral density of y.t/ consists of point masses on
g f g
thediscreteunitcircleT2N $ N 1;$ n 2;:::;$N ,where
WDf ! C ! C g
$ eik"=N: (4)
k
D
Moreprecisely,definethefunction
N
˚.$/ c $ k (5)
k !
D
k N 1
DX! C
on T2N. This is the discrete Fourier transform (DFT) of the sequence
.c ;:::;c /,whichcanberecoveredbytheinverseDFT
N 1 N
! C
1 N "
c ˚.$ /$k eik#˚.ei#/d%; (6)
k D 2N j j D
jDX!NC1 Z!"
284 A.LindquistandG.Picci
where! isastepfunctionwithsteps 1 ateach" ;i.e.,
2N k
N
d#
d!.#/ ı.ei# "/ : (7)
j
D ! 2N
j N 1
DX! C
Consequently,by(3),dF.#/ ˚.ei#/d!.#/.Wenoteinpassingthat
D
$
eik#d!.#/ ı ; (8)
k0
D
Z $
!
whereı equalsonefork 0andzerootherwise.Toseethis,notethat,fork 0,
k0
D ¤
$ 1 N
.1!"k/ eik#d! D 2N "kj !"kjC1
Z!$ jDX!NC1! "
1
" N 1 "N 1 0:
D 2N k! C ! kC D
# $
Since y.t/ isstationaryandfullrank,theToeplitzmatrix
f g
c c c c
0 1 2 n
N N """ N
c c c c
1 0 1 n 1
2 N """N ! 3
Tn D66c:::2 c:::1 c:::0 "::":"Ncn:::!277 (9)
6 7
6c c c c 7
64 n n!1 n!2 """ 0 75
is positive definite for all n Z. However, this condition is not sufficient for
2
c ;c ;:::;c tobeabona-fidecovariancesequenceofaperiodicprocess,ascanbe
0 1 n
seenfromthefollowingsimpleexample.Considerareal-valuedperiodicstationary
processyofperiodfour.Then
y.1/ c c c c
0 1 2 3
y.2/ c c c c
E82 3 y.1/y.2/y.3/y.4/ 9 2 1 0 1 23:
ˆˆ<6yy..34//7% &>>=D6cc2 cc1 cc0 cc17
6 7 6 3 2 1 07
ˆˆ:4 5 >>; 4 5
ModelingofStationaryPeriodicTimeSeriesbyARMARepresentations 285
Thenlookingatthecovariancematrixfortwoperiods,weobtain
c c c c c c c c
0 1 2 3 0 1 2 3
c c c c c c c c
2 1 0 1 2 1 0 1 23
y.1/ c c c c c c c c
2 1 0 1 2 1 0 1
y.2/ 6c c c c c c c c 7
E8ˆˆˆ<26 ::: 37 y.1/y.2/!!! y.8/ 9>>>=D6666c30 c21 c12 c03 c30 c21 c12 c037777;
6y.8/7! " 6c1 c0 c1 c2 c1 c0 c1 c27
6 7 6 7
ˆˆˆ:4 5 >>>; 666cc2 cc1 cc0 cc1 cc2 cc1 cc0 cc1777
6 3 2 1 0 3 2 1 07
4 5
whichisaToeplitzmatrixonlywhenc c .Thereforethecondition c c is
3 1 3 1
D D
necessary.Consequently
c c c c c c c c
0 1 2 1 0 1 2 1
c c c c c c c c
2 1 0 1 2 1 0 1 23
c c c c c c c c
2 1 0 1 2 1 0 1
6c c c c c c c c 7
T 6 1 2 1 0 1 2 1 07
8 6 7
D6c0 c1 c2 c1 c0 c1 c2 c17
6 7
6c1 c0 c1 c2 c1 c0 c1 c27
6 7
6c2 c1 c0 c1 c2 c1 c0 c17
6 7
6c c c c c c c c 7
6 1 2 1 0 1 2 1 07
4 5
is a circulant matrix, where the columns are shifted cyclically, the last component
movedtothetop.Circulantmatriceswillplayakeyroleinthefollowing.
3 The CovarianceExtensionProblemforPeriodicProcesses
Supposethatwearegivenapartialcovariancesequencec ;c ;:::;c withn < N
0 1 n
suchthattheToeplitzmatrixT ispositivedefinite.Considertheproblemoffinding
n
andextensionc ;c ;:::;c sothatthecorrespondingsequencec ;c ;:::;c
n 1 n 2 N 0 1 N
isthecovarianceCsequeCnceofastationaryprocessofperiod2N.
Ingeneralthisproblemwillhaveinfinitelymanysolutions,and,forreasonsthat
willbecomeclearlater,weshallrestrictourattentiontospectralfunctions(5)which
arerationalinthesensethat
P.!/
˚.!/ ; (10)
D Q.!/
wherePandQareHermitianpseudo-polynomialsofdegreeatmostn,thatisofthe
form
n
P.!/ p ! k; p p : (11)
k ! k k
D ! D N
k n
XD!
286 A.LindquistandG.Picci
Let P .N/ be the cone of all pseudo-polynomials (11) that are positive on
the discreCte unit circle T2N, and let P P .N/ be the subset of pseudo-
polynomials (11) such that P.ei!/ > 0 fCor a!ll ! C Π";"#. Moreover let C .N/
bethedualconeofallpartialcovariancesequences2c " .c ;c ;:::;c /suchtChat
0 1 n
D
n
c;p c p >0 forallP P .N/ 0 ;
k k
h iWD N 2 C nf g
k n
XD!
andletC bedefinedinthesamewayasthedualconeofP .Itcanbeshown[25]
that c CC is equivalent to the Toeplitz condition T > 0.CSince P P .N/,
n
wehav2eCC.N/ C ,soingeneralc C .N/isastricterconditionCtha!nTC>0.
n
TheproCofoft!hefCollowingtheorem2canCbefoundin[30].
Theorem1. Let c C .N/. Then, for each P P .N/, there is a unique Q
P .N/suchthat 2 C 2 C 2
C
P
˚
D Q
satisfiesthemomentconditions
"
eik!˚.ei!/d$.!/ c ; k 0;1;:::;n: (12)
k
D D
Z "
!
Consequently the family of solutions (10) of the covariance extension problem
stated above are parameterized by P P .N/ in a bijective fashion. From the
followingtheoremweseethat,forany2P CP .N/,thecorrespondinguniqueQ
P .N/canbeobtainedbyconvexoptimi2zatioCn.Wereferthereaderto[30]forth2e
prCoofs.
Theorem2. Letc C .N/andP P .N/.Thentheproblemtomaximize
2 C 2 C
"
IP.˚/ P.ei!/log˚.ei!/d$ (13)
D
Z "
!
subjecttothemomentconditions(12)hasauniquesolution,namely(10),whereQ
istheuniqueoptimalsolutionoftheproblemtominimize
"
JP.Q/ c;q P.ei!/logQ.ei!/d$ (14)
Dh i"
Z "
!
over all Q P .N/, where q .q0;q1;:::;qn/. The functional JP is strictly
convex. 2 C WD
ModelingofStationaryPeriodicTimeSeriesbyARMARepresentations 287
Theorems 1 and 2 are discrete versions of corresponding results in [6, 9]. The
solutioncorrespondingtoP 1iscalledthemaximum-entropysolutionbyvirtue
D
of(13).
Remark3. As N the process y looses it periodic character, and its spectral
! 1
density˚ becomescontinuousanddefinedonthewholeunitcirclesothat
1
! d"
eik"˚ .ei"/ c ; k 0;1;:::;n: (15)
k
Z ! 1 2! D D
!
Infact,denotingbyQ thesolutionofTheorem1,itwasshownin[30]that˚
N
P=Q ,where,foreachfixedP, 1 D
1
Q lim Q
N
1 DN
!1
istheuniqueQsuchthat˚ P=Qsatisfiesthemomentconditions(15).
1 D
4 ReformulationinTerms ofCirculantMatrices
Circulantmatrices[14]areToeplitzmatriceswithaspecialcirculantstructure
# # # #
0 $ $ 1 1
! !!!
# # # #
1 0 $ 2
2 !!! 3
Circf#0;#1;:::;#$gD66#:::2 #:::1 #:::0 !::!:! #:::377; (16)
6 7
6# # # # 7
64 $ $!1 $!2 !!! 075
where the columns (or, equivalently, rows) are shifted cyclically, and where
# ;# ;:::;# here are taken to be complex numbers. In our present covariance
0 1 $
extensionproblemweconsiderHermitiancirculantmatrices
M Circ m ;m ;m ;:::;m ;m ;:::;m ;m ; (17)
0 1 2 N N 1 2 1
WD f N ! N N g
whichcanberepresentedinform
N
M m S k; m m (18)
k ! k k
D ! D N
k N 1
DX! C
288 A.LindquistandG.Picci
whereSisthenonsingular2N 2N cyclicshiftmatrix
!
010 0 :::0
001 0 :::0
2 3
000 1 :::0
SWD66::: ::: ::: ::: ::: :::77: (19)
6 7
6 7
6000 0 0 17
6 7
6100 0 0 07
6 7
4 5
Thepseudo-polynomial
N
M.!/ m ! k; m m (20)
k ! k k
D ! D N
k N 1
DX! C
is called the symbol of M. Clearly S is itself a circulant matrix (although not
Hermitian)withsymbolS.!/ !.Anecessaryandsufficientconditionforamatrix
D
Mtobecirculantisthat
SMST M: (21)
D
Hence, since S 1 ST, the inverse of a circulant matrix is also circulant. More
!
D
generally, if A and B are circulant matrices of the same dimension with symbols
A.!/ and B.!/ respectively, then AB and A B are circulant matrices with
C
symbols A.!/B.!/ and A.!/ B.!/, respectively. In fact, the circulant matrices
C
of a fixed dimension form an algebra—more precisely, a commutative *-algebra
with the involution * being the conjugate transpose—and the DFT is an algebra
homomorphism of the set of circulant matrices onto the pseudo-polynomials of
degree at most N in the variable ! T2N. Consequently, circulant matrices
2
commute,and,ifMisacirculantmatrixwithsymbolM.!/,thenM 1 iscirculant
!
withsymbolM.!/ 1.
!
Theproofofthefollowingpropositionisimmediate.
Proposition4. Let y.t/ t N 1;:::;N beastationaryprocesswithperiod
f I D" C g
2N andcovariancelags(3),andletybethe2N-dimensionalstochasticvectory
D
Œy. N 1/;y. N 2/; ;y.N/"T.Then,with denotingconjugatetranspose,
"
" C " C ###
˙ E yy" Circ c0;c1;c2;:::;cN;cN 1;:::;c2;c1 (22)
WD f gD f N ! N N g
isa2N 2N Hermitiancirculantmatrixwithsymbol˚.!/ givenby(5).
!
The covariance extension problem of Sect.3, called the circulant rational
covariance extension problem, can now be reformulated as a matrix extension
problem. The given covariance data c .c ;c ;:::;c / can be represented as a
0 1 n
D
circulantmatrix
C Circ c ;c ;:::;c ;0;:::;0;c ;c ;:::;c (23)
0 1 n n n 1 1
D f N N ! N g
ModelingofStationaryPeriodicTimeSeriesbyARMARepresentations 289
withsymbol
n
C.!/ c ! k; (24)
k !
D
k n
XD!
where the unknown covariance lags c ;c ;:::;c in (22), to be determined,
n 1 n 2 N
herearereplacedbyzeros.AcirculantmCatrixCoftype(23)iscalledbandedoforder
n.Werecallthatn < N.Fromnowonewedroptheattribute‘Hermitian’sincewe
shallonlyconsidersuchcirculantmatricesinthesequel.Abandedcirculantmatrix
ofordernwillthusbedeterminedbyn 1(complex)parameters.
C
Thenextlemmaestablishestheconnectionbetweencirculantmatricesandtheir
symbols.
Lemma5. LetMbeacirculantmatrixwithsymbolM.!/.Then
M F diag M.! /;M.! /;:::;M.! / F; (25)
" N 1 N 2 N
D ! C ! C
! "
whereFistheunitarymatrix
!N 1 !N 2 ! N
N! 1 N! 1 !!! !N 1
!:C !:C !:C
: : :
2 : : : 3
1 !!!
F !N 1 !N 2 ! N : (26)
D p2N 66 0::! 0::! !!! 0!:: 77
6 : : : 7
6 !!! 7
6 !N 1 !N 2 ! N 7
6 N! N! !!! N! 7
4 5
Moreover,ifM.! />0forallk,then
k
logM F diag logM.! /;logM.! /;:::;logM.! / F: (27)
" N 1 N 2 N
D ! C ! C
! "
Proof. ThediscreteFouriertransformF mapsasequence.g ;g ;:::;g /
N 1 N 2 N
intothesequenceofcomplexnumbers ! C ! C
N
G.!/ g ! k; j N 1; N 2;:::;N: (28)
j WD k j! D" C " C
k N 1
DX! C
ThesequencegcanberecoveredfromGbytheinversetransform
"
g eik#G.ei#/d$.#/; k N 1; N 2;:::;N: (29)
k
D D" C " C
Z "
!
Thiscorrespondencecanbewritten
g Fg; (30)
O D
290 A.LindquistandG.Picci
wnohnesrienggOuWlDar2.N2N/!212NGV.a!n!dNeCrm1/o;n:d:e:;mGa.t!rNix/(T2,6g).WCDle.agrl!yNFC1i;s:u:n:i;tgarNy/.T,SainncdeF is the
! ! "
N
Mg m S k
k !
D
k N 1
DX! C
andŒS kg" g ,whereg g ,wehave
! j j k k 2N k
D ! C D
N N
F.Mg/ ! j m g
! k j k
D !
j N 1 k N 1
DX! C DX! C
N N
m ! k g ! .j k/ M.!/Fg;
k ! j k ! !
D ! D
k N 1 j N 1
DX! C DX! C
whichyields
p2N.FMg/ M.!/p2N.Fg/; j N 1; N 2;:::;N;
j j j
D D" C " C
fromwhich(25)follows.Finally,since,asafunctionofz C,logM.z/isanalytic
2
in the neighborhood of each M.! / > 0,theeigenvalues of logMare justthe real
k
numbers logM.! /, k N 1;:::;N, by the spectral mapping theorem [16,
k
D" C
p.557],andhence(27)follows.
WearenowinapositiontoreformulateTheorems1and2intermsofcirculant
matrices. To this end first note that, in view of Lemma 5, the cone P .N/
corresponds to the class of positive-definite banded 2N 2N circulant matCrices
!
P of order n. Moreover, by Plancherel’s Theorem for DFT, which is a simple
consequenceof(8),wehave
n N
1
c p C.!/P.!/;
k k j j
N D 2N
k n j N 1
XD! DX! C
andhence,byLemma5,
1
c;p tr.CP/; (31)
h iD 2N
wheretrdenotestrace.
Consequently, c C .N/ if and only if tr.CP/ > 0 for all nonzero, positive-
semidefinite,banded22NC 2NcirculantmatricesPofordern.Moreover,ifQandP
!
arecirculantmatriceswithsymbolsP.!/andQ.!/,respectively,then,byLemma5,
P.!/=Q.!/ is the symbol of Q 1P. Therefore Theorem 1 has the following matrix
!
version.