Table Of ContentOperator Theory: Advances and Applications
Vol. 204
Editor:
I. Gohberg
Editorial Office:
M. Putinar (Santa Barbara, CA, USA)
School of Mathematical Sciences A.C.M. Ran (Amsterdam, The Netherlands)
Tel Aviv University L. Rodman (Williamsburg, VA, USA)
Ramat Aviv J. Rovnyak (Charlottesville, VA, USA)
Israel B.-W. Schulze (Potsdam, Germany)
F. Speck (Lisboa, Portugal)
I.M. Spitkovsky (Williamsburg, VA, USA)
Editorial Board:
S. Treil (Providence, RI, USA)
D. Alpay (Beer Sheva, Israel) C. Tretter (Bern, Switzerland)
J. Arazy (Haifa, Israel) H. Upmeier (Marburg, Germany)
A. Atzmon (Tel Aviv, Israel) N. Vasilevski (Mexico, D.F., Mexico)
J.A. Ball (Blacksburg, VA, USA) S. Verduyn Lunel (Leiden, The Netherlands)
H. Bart (Rotterdam, The Netherlands) D. Voiculescu (Berkeley, CA, USA)
A. Ben-Artzi (Tel Aviv, Israel) D. Xia (Nashville, TN, USA)
H. Bercovici (Bloomington, IN, USA) D. Yafaev (Rennes, France)
A. Böttcher (Chemnitz, Germany)
K. Clancey (Athens, GA, USA)
R. Curto (Iowa, IA, USA) Honorary and Advisory Editorial Board:
K. R. Davidson (Waterloo, ON, Canada) L.A. Coburn (Buffalo, NY, USA)
M. Demuth (Clausthal-Zellerfeld, Germany) H. Dym (Rehovot, Israel)
A. Dijksma (Groningen, The Netherlands) C. Foias (College Station, TX, USA)
R. G. Douglas (College Station, TX, USA) J.W. Helton (San Diego, CA, USA)
R. Duduchava (Tbilisi, Georgia) T. Kailath (Stanford, CA, USA)
A. Ferreira dos Santos (Lisboa, Portugal) M.A. Kaashoek (Amsterdam, The Netherlands)
A.E. Frazho (West Lafayette, IN, USA) P. Lancaster (Calgary, AB, Canada)
P.A. Fuhrmann (Beer Sheva, Israel) H. Langer (Vienna, Austria)
B. Gramsch (Mainz, Germany) P.D. Lax (New York, NY, USA)
H.G. Kaper (Argonne, IL, USA) D. Sarason (Berkeley, CA, USA)
S.T. Kuroda (Tokyo, Japan) B. Silbermann (Chemnitz, Germany)
L.E. Lerer (Haifa, Israel) H. Widom (Santa Cruz, CA, USA)
B. Mityagin (Columbus, OH, USA)
V. Olshevski (Storrs, CT, USA)
Subseries Joseph A. Ball
Linear Operators and Linear Systems Department of Mathematics
Virginia Tech
Subseries editors: Blacksburg, VA 24061
USA
Daniel Alpay
Department of Mathematics André M.C. Ran
Ben Gurion University of the Negev Division of Mathematics and
Beer Sheva 84105 Computer Science
Israel Faculty of Sciences
Vrije Universiteit
NL-1081 HV Amsterdam
The Netherlands
An Operator
Perspective on
Signals and Systems
Arthur E. Frazho
Wisuwat Bhosri
Birkhäuser
Basel · Boston · Berlin
Authors:
Arthur E. Frazho Wisuwat Bhosri
School of Aeronautics and Astronautics 541/15 Lamphun Road
Purdue University Amphoe Muang
West Lafayette, IN 47907-2045 Chiang Mai 50000
USA Thailand
e-mail: [email protected] e-mail: [email protected]
2000 Mathematics Subject Classification: 47A20, 47A57, 47B35
Library of Congress Control Number: 2009938997
Bibliographic information published by Die Deutsche Bibliothek.
Die Deutsche Bibliothek lists this publication in the Deutsche Nationalbibliografie;
detailed bibliographic data is available in the Internet at http://dnb.ddb.de
ISBN 978-3-0346-0291-4 Birkhäuser Verlag AG, Basel – Boston – Berlin
This work is subject to copyright. All rights are reserved, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, re-use of
illustrations, recitation, broadcasting, reproduction on microfilms or in other ways, and storage
in data banks. For any kind of use permission of the copyright owner must be obtained.
© 2010 Birkhäuser Verlag AG
Basel · Boston · Berlin
P.O. Box 133, CH-4010 Basel, Switzerland
Part of Springer Science+Business Media
Printed on acid-free paper produced from chlorine-free pulp. TCF∞
Printed in Germany
ISBN 978-3-0346-0291-4 e-ISBN 978-3-0346-0292-1
9 8 7 6 5 4 3 2 1 www.birkhauser.ch
Preface
In this monograph, we combine operator techniques with state space methods
to solve factorization, spectral estimation, and interpolation problems arising in
control and signal processing. We present both the theory and algorithms with
some Matlab code to solve these problems.
A classical approach to spectral factorization problems in control theory is
based on Riccati equations arising in linear quadratic controltheory and Kalman
filtering. One advantage of this approach is that it readily leads to algorithms
in the non-degenerate case. On the other hand, this approach does not easily
generalize to the nonrational case, and it is not always transparent where the
Riccati equations are coming from.
Operator theory has developed some elegant methods to prove the existence
of a solution to some of these factorization and spectral estimation problems in a
verygeneralsetting.However,these techniques are in generalnotusedto develop
computational algorithms. In this monograph, we will use operator theory with
statespacemethodstoderivecomputationalmethods tosolvefactorization,spec-
tralestimation,andinterpolationproblems.Itis emphasizedthatourapproachis
geometric and the algorithms are obtained as a special application of the theory.
We will present two methods for spectral factorization. One method derives algo-
rithms based on finite sections of a certain Toeplitz matrix. The other approach
uses operator theory to develop the Riccati factorization method. Finally, we use
isometric extension techniques to solve some interpolation problems.
The monograph is divided into five parts. In the first part, we present some
classicalresults fromoperatortheory.This includes the Wolddecomposition,uni-
lateral and bilateral shifts, the Beurling-Lax-Halmos Theorem, and the Naimark
representation Theorem. Chapter 5 on the Naimark representation Theorem is
one of the fundamental tools that is used throughout the monograph.The reader
familiar with operator theory can skip this part and refer back to it as needed.
This part also includes some results on rational functions which are not usually
presented in elementary operator theory. The first part is self contained and is
written for someone with a minimal background in operator theory. The other
four parts aremore or less independent ofeachother, andcanbe readseparately.
There may be some cross references. However, this should not cause any major
difficulty.
In part II, we develop finite section techniques to compute the inner-outer
factorization of rational functions and solve a spectral factorization problem in
both the squareand non-squarecases.Furthermore, operatortechniques are used
to solve some sinusoid estimation problems in signal processing. In particular,
we use geometric methods to develop the Capon-Genorimus sinusoid estimation
algorithm.Finite section methods are also used to solve some sinusoid estimation
problems. Many of these techniques are based on the Levinson and Kalman-Ho
algorithm. Several examples using Matlab are given.
vi Preface
In part III, we use Riccati techniques to solve factorization and Darlington
synthesis problems. These Riccati techniques are developed from the Naimark
representationTheorem.Chapter11is devotedto theKalmanfilter.This chapter
can be read independently from the rest of the monograph. This is included to
demonstrate where the Riccati equations in control theory originally came from,
and how they can be used to solve Kalman and Wiener filtering problems.
ThefourthpartisanintroductiontopositiverealandH∞typeinterpolation
problems.Our approachis basedon extending a contractionto an isometry.Here
we do not present the set of all solutions, we just give the central solution and
corresponding state space formula. The central solution is the one that is most
widely used in applications.
The fifth part is the appendix which includes a short review of state space
techniques used throughout the monograph. We also place a special emphasis
on the Kalman-Ho algorithm, which plays a fundamental role in some of our
computationaltechniques.The last chapter is devotedto the Levinsonalgorithm.
Finally, the Gohberg-Semencul-Heinig inversion formula for a positive Toeplitz
operator is presented.
It is assumed that the reader is familiar with linear algebra, and some ele-
mentary facts from operator theory such as the projection theorem, the adjoint
of an operator and a positive operator. Our approach is geometric and we do not
relyonmeasuretheoretictechniques.Itisalsoassumedthatthe readeris familiar
with some elementary concepts fromlinear systems theory suchas controllability,
observability and state space realization. A review of some of these state space
techniques is given in the appendix. Some sections can be skipped without any
loss of continuity and the reader will be notified when this is the case. We have
alsousedthenotesattheendofthechaptertodevelopsometechnicalconnections
between our results and some of the existing theory. Finally, it is noted that we
developed our theory using Hardy spaces of functions which are analytic outside
the open unit disc. This was done mainly because these Hardy spaces are more
naturallysuitedtothefastFouriertransformalgorithminMatlabandstatespace
methods.
We hope that this monograph is beneficial to both mathematicians and en-
gineers. We believe that the operator theoretic foundation provides additional
insightintospectralfactorizationandsignalprocessing.Moreover,thisframework
may be useful for other engineering applications. Finally, the applications and
examples may provide some additional insight into other mathematical problems.
August, 2009 The authors
Contents
Preface v
I Basic Operator Theory 1
1 The Wold Decomposition 3
1.1 The Unilateral Shift . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 The Eigenvalues for the BackwardShift . . . . . . . . . . . . . . . 6
1.3 The Wold Decomposition . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 The Bilateral Shift . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.5 Abstract Toeplitz Operators . . . . . . . . . . . . . . . . . . . . . . 16
1.5.1 Abstract Toeplitz operators viewed as a compression . . . . 19
1.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2 Toeplitz and Laurent Operators 23
2.1 The Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.1.1 The subspace L2(E) . . . . . . . . . . . . . . . . . . . . . . 24
+
2.2 Hardy Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3 Multiplication Operators . . . . . . . . . . . . . . . . . . . . . . . . 29
2.4 Laurent Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.5 Toeplitz Operators and Matrices . . . . . . . . . . . . . . . . . . . 34
2.6 Toeplitz Matrices and H∞ Functions . . . . . . . . . . . . . . . . . 36
2.7 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3 Inner and Outer Functions 41
3.1 The Beurling-Lax-HalmosTheorem . . . . . . . . . . . . . . . . . . 41
3.1.1 The invariant subspaces for the backward shift . . . . . . . 43
3.2 Inner-Outer Factorizations . . . . . . . . . . . . . . . . . . . . . . . 45
3.3 Invertible Outer Functions . . . . . . . . . . . . . . . . . . . . . . . 49
3.4 The Determinant of Inner and Outer Functions . . . . . . . . . . . 51
3.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
viii Contents
4 Rational Inner and Outer Functions 55
4.1 Blaschke Products . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.2 State Space Realizations for Rational Inner Functions . . . . . . . 58
4.3 Rational Two-Sided Inner Functions . . . . . . . . . . . . . . . . . 64
4.4 Rational Outer Functions . . . . . . . . . . . . . . . . . . . . . . . 68
4.5 Inner-Outer Factorization and McMillan Degree. . . . . . . . . . . 72
4.6 Inner-Outer Factorization and Finite Sections . . . . . . . . . . . . 78
4.7 The Douglas-Shapiro-Shields Factorization. . . . . . . . . . . . . . 81
4.8 The Douglas-Shapiro-Shields Factorization
for Rational Functions . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.8.1 A Factorization for rational L∞ functions . . . . . . . . . . 86
4.9 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5 The Naimark Representation 91
5.1 The Naimark Representation Theorem . . . . . . . . . . . . . . . . 92
5.2 The Maximal Outer Spectral Factor . . . . . . . . . . . . . . . . . 97
5.2.1 The eigenspace for the unitary part . . . . . . . . . . . . . 102
5.3 The Inner-Outer Factorization Revisited . . . . . . . . . . . . . . . 103
5.4 Positive Real Functions . . . . . . . . . . . . . . . . . . . . . . . . 106
5.5 Minimal Isometric Liftings . . . . . . . . . . . . . . . . . . . . . . . 110
5.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
6 The Rational Case 117
6.1 Rational Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
6.2 The Positive Real Lemma . . . . . . . . . . . . . . . . . . . . . . . 123
6.3 Finite Dimensional Unitary Part . . . . . . . . . . . . . . . . . . . 125
6.4 A Classical Ergodic Result. . . . . . . . . . . . . . . . . . . . . . . 127
6.5 Another Ergodic Result . . . . . . . . . . . . . . . . . . . . . . . . 130
6.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
II Finite Section Techniques 143
7 The Levinson Algorithm and Factorization 145
7.1 The Case when T is Invertible . . . . . . . . . . . . . . . . . . . . 145
R
7.2 The Classical Schur Inversion Formula . . . . . . . . . . . . . . . . 148
7.3 The Schur Complement and Toeplitz Matrices. . . . . . . . . . . . 149
7.4 Schur Complement and Maximal Outer Factor . . . . . . . . . . . 151
7.5 Carath´eodoryInterpolation . . . . . . . . . . . . . . . . . . . . . . 155
7.6 A Finite Sections Approach to Factorization . . . . . . . . . . . . . 159
7.6.1 A finite section approach to outer factorization . . . . . . . 160
7.7 An Inner-Outer Factorization Procedure . . . . . . . . . . . . . . . 162
7.8 Rational Contractive Analytic Functions . . . . . . . . . . . . . . . 165
7.8.1 A contractive realization procedure . . . . . . . . . . . . . . 169
Contents ix
7.9 A Spectral Factorization Approach to Filtering . . . . . . . . . . . 171
7.10 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
8 Isometric Representations and Factorization 179
8.1 Isometric Representations and Finite Sections . . . . . . . . . . . . 179
8.2 A Basic Optimization Problem . . . . . . . . . . . . . . . . . . . . 185
8.3 The Lower Triangular Cholesky Factorization . . . . . . . . . . . . 188
8.4 Cholesky Factorization and Maximal
Outer Spectral Factors . . . . . . . . . . . . . . . . . . . . . . . . . 194
8.5 Some Examples of the Cholesky Factorization . . . . . . . . . . . . 197
8.5.1 Scalar-valued outer function . . . . . . . . . . . . . . . . . . 197
8.5.2 Multi-input multi-output square outer function . . . . . . . 198
8.5.3 Non-square outer function . . . . . . . . . . . . . . . . . . . 199
8.5.4 Non-square outer function and a unitary part . . . . . . . . 201
8.6 An Inner-Outer Factorization Algorithm . . . . . . . . . . . . . . . 202
8.6.1 A scalar inner-outer factorization example . . . . . . . . . . 203
8.6.2 A non-square inner-outer factorization example . . . . . . . 204
8.6.3 An outer function and a unitary part. . . . . . . . . . . . . 205
8.7 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
9 Signal Processing 209
9.1 A Fundamental Optimization Problem . . . . . . . . . . . . . . . . 209
9.2 Sinusoid Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . 213
9.3 Sinusoid Estimation: Capon-Geronimus . . . . . . . . . . . . . . . 221
9.4 A Nested Optimization Problem . . . . . . . . . . . . . . . . . . . 223
9.5 Limit Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
9.6 The Outer Function K (z,α) . . . . . . . . . . . . . . . . . . . . . 233
n
9.7 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
III Riccati Methods 245
10 Riccati Equations and Factorization 247
10.1 Algebraic Riccati Equations . . . . . . . . . . . . . . . . . . . . . . 247
10.2 The Case when T is Strictly Positive . . . . . . . . . . . . . . . . 253
R
10.3 The Riccati Difference Equation . . . . . . . . . . . . . . . . . . . 256
10.4 A Riccati Approach to Inner-Outer Factorizations . . . . . . . . . 260
10.5 The Outer Factor for γ2I −G∗G . . . . . . . . . . . . . . . . . . . 265
10.6 Darlington Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . 268
10.7 Riccati Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
10.7.1 The minimum principle . . . . . . . . . . . . . . . . . . . . 274
10.7.2 Proof of Theorem 10.7.1 . . . . . . . . . . . . . . . . . . . . 276
10.8 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
