Fourier Analysis and Stochastic Processes

396 Pages·2014·4.34 MB·English

by  Pierre Brémaud (auth.)

Universitext
Pierre Brémaud
Fourier
Analysis and
Stochastic
Processes
Universitext
Universitext
Series editors
Sheldon Axler
San Francisco State University, San Francisco, CA, USA
Vincenzo Capasso
Università degli Studi di Milano, Milan, Italy
Carles Casacuberta
Universitat de Barcelona, Barcelona, Spain
Angus MacIntyre
Queen Mary University of London, London, UK
Kenneth Ribet
University of California, Berkeley, CA, USA
Claude Sabbah
CNRS École Polytechnique Centre de mathématiques, Palaiseau, France
Endre Süli
University of Oxford, Oxford, UK
Wojbor A. Woyczynski
Case Western Reserve University, Cleveland, OH, USA
Universitext is a series of textbooks that presents material from a wide variety of
mathematical disciplines at master’s level and beyond. The books, often well
class-tested by their author, may have an informal, personal, even experimental
approachtotheirsubjectmatter.Someofthemostsuccessfulandestablishedbooks
intheserieshaveevolvedthroughseveraleditions,alwaysfollowingtheevolution
of teaching curricula, into very polished texts.
Thus as research topics trickle down into graduate-level teaching, first textbooks
written for new, cutting-edge courses may make their way into Universitext.
More information about this series at http://www.springer.com/series/223
é
Pierre Br maud
Fourier Analysis
and Stochastic Processes
123
Pierre Brémaud
Inria
Paris
France
and
ÉcolePolytechnique Fédérale de Lausanne
Lausanne
Switzerland
ISSN 0172-5939 ISSN 2191-6675 (electronic)
ISBN 978-3-319-09589-9 ISBN 978-3-319-09590-5 (eBook)
DOI 10.1007/978-3-319-09590-5
LibraryofCongressControlNumber:2014946193
MathematicsSubjectClassification:42A38,42B10,60G10,60G12,60G35,60G55
SpringerChamHeidelbergNewYorkDordrechtLondon
©SpringerInternationalPublishingSwitzerland2014
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionor
informationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodology now known or hereafter developed. Exempted from this legal reservation are brief
excerpts in connection with reviews or scholarly analysis or material supplied specifically for the
purposeofbeingenteredandexecutedonacomputersystem,forexclusiveusebythepurchaserofthe
work. Duplication of this publication or parts thereof is permitted only under the provisions of
theCopyright Law of the Publisher’s location, in its current version, and permission for use must
always be obtained from Springer. Permissions for use may be obtained through RightsLink at the
CopyrightClearanceCenter.ViolationsareliabletoprosecutionundertherespectiveCopyrightLaw.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt
fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Pour Marion
Contents
1 Fourier Analysis of Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Fourier Theory in L1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Fourier Transform and Fourier Series. . . . . . . . . . . . . . . 1
1.1.2 Convergence Theory . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.1.3 The Poisson Summation Formula . . . . . . . . . . . . . . . . . 37
1.2 Z-Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
1.2.1 Fourier Sums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
1.2.2 Transfer Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
1.3 Hilbert Spaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
1.3.1 Isometric Extension. . . . . . . . . . . . . . . . . . . . . . . . . . . 55
1.3.2 Orthogonal Projection . . . . . . . . . . . . . . . . . . . . . . . . . 61
1.3.3 Orthonormal Expansions . . . . . . . . . . . . . . . . . . . . . . . 70
1.4 Fourier Theory in L2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
1.4.1 Fourier Transform in L2. . . . . . . . . . . . . . . . . . . . . . . . 73
1.4.2 Fourier Series in L2 . . . . . . . . . . . . . . . . . . . . . . . . . . 81
loc
1.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
2 Fourier Theory of Probability Distributions. . . . . . . . . . . . . . . . . . 91
2.1 Characteristic Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
2.1.1 Basic Facts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
2.1.2 Inversion Formula. . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
2.1.3 Gaussian Vectors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
2.2 Convergence in Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . 103
2.2.1 Paul Lévy’s Theorem. . . . . . . . . . . . . . . . . . . . . . . . . . 103
2.2.2 Bochner’s Theorem. . . . . . . . . . . . . . . . . . . . . . . . . . . 108
2.3 Weak Convergence of Finite Measures. . . . . . . . . . . . . . . . . . . 111
2.3.1 Helly’s Theorem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
2.3.2 Proof of Paul Lévy’s Theorem . . . . . . . . . . . . . . . . . . . 113
2.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
vii
viii Contents
3 Fourier Analysis of Stochastic Processes . . . . . . . . . . . . . . . . . . . . 119
3.1 Review of Stochastic Processes. . . . . . . . . . . . . . . . . . . . . . . . 119
3.1.1 Distribution of a Stochastic Process. . . . . . . . . . . . . . . . 119
3.1.2 Measurability and Sample Path Properties . . . . . . . . . . . 131
3.2 Gaussian Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
3.2.1 Gaussian Subspaces. . . . . . . . . . . . . . . . . . . . . . . . . . . 134
3.2.2 Brownian Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
3.2.3 The Wiener–Doob Integral. . . . . . . . . . . . . . . . . . . . . . 137
3.3 The Power Spectral Measure. . . . . . . . . . . . . . . . . . . . . . . . . . 141
3.3.1 The Covariance Function . . . . . . . . . . . . . . . . . . . . . . . 141
3.3.2 Fourier Representation of the Covariance. . . . . . . . . . . . 144
3.3.3 Filtering and White Noise . . . . . . . . . . . . . . . . . . . . . . 147
3.3.4 Long-range Dependence and Self-similarity . . . . . . . . . . 153
3.4 Fourier Analysis of the Trajectories . . . . . . . . . . . . . . . . . . . . . 157
3.4.1 The Cramér–Khinchin Decomposition . . . . . . . . . . . . . . 157
3.4.2 Linear Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
3.4.3 Multivariate WSS Stochastic Processes. . . . . . . . . . . . . . . 169
3.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
4 Fourier Analysis of Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . 181
4.1 Power Spectral Measure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
4.1.1 Herglotz’s Theorem. . . . . . . . . . . . . . . . . . . . . . . . . . . 181
4.1.2 The Periodogram. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
4.1.3 The Cramér–Khinchin Decomposition . . . . . . . . . . . . . . 196
4.2 ARMA Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
4.2.1 Prediction Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
4.2.2 Regular ARMA Processes. . . . . . . . . . . . . . . . . . . . . . . . 208
4.2.3 Long-Range Dependence . . . . . . . . . . . . . . . . . . . . . . . 214
4.3 General Prediction Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
4.3.1 Wold’s Decomposition. . . . . . . . . . . . . . . . . . . . . . . . . 224
4.3.2 Innovations Sequence . . . . . . . . . . . . . . . . . . . . . . . . . 226
4.3.3 Kolmogorov’s Regularity Criterion . . . . . . . . . . . . . . . . 230
4.4 Realization Theory for Spectral Analysis . . . . . . . . . . . . . . . . . 233
4.4.1 The Levinson–Durbin Algorithm. . . . . . . . . . . . . . . . . . 233
4.4.2 Maximum Entropy Realization . . . . . . . . . . . . . . . . . . . 244
4.4.3 Pisarenko’s Realization . . . . . . . . . . . . . . . . . . . . . . . . 247
4.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
5 Power Spectra of Point Processes . . . . . . . . . . . . . . . . . . . . . . . . . 259
5.1 Review of Point Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
5.1.1 The Distribution of a Point Process. . . . . . . . . . . . . . . . 260
5.1.2 Poisson Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
5.1.3 Renewal Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
Contents ix
5.2 Power Spectral Measure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
5.2.1 Covariance Measure and Bartlett Spectrum. . . . . . . . . . . 289
5.2.2 Transformed Point Processes. . . . . . . . . . . . . . . . . . . . . 307
5.2.3 Pulse Amplitude Modulation. . . . . . . . . . . . . . . . . . . . . 317
5.3 Random Sampling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
5.3.1 Spectral Measure of the Sample Brush. . . . . . . . . . . . . . 322
5.3.2 Reconstruction from Samples . . . . . . . . . . . . . . . . . . . . 328
5.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
Appendix A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
Introduction
A unified treatment of all the aspects of Fourier theory relevant to the Fourier
analysis of stochastic processes is not only unavoidable, but also intellectually
satisfying, and in fact time-saving. This is why this book contains the classical
Fourier theory of functions (Fourier series and transforms in L1 and L2,
z-transforms), the Fourier theory of probability distributions (characteristic
functions, convergence in distribution) and the elements of Hilbert space theory
(orthogonal projectionprinciple,orthonormalbases,isometricextensiontheorem)
whichareindispensableprerequisitesfortheFourieranalysisofstochasticprocesses
(randomfields,timeseriesandpointprocesses)ofthelastthreechapters.
Stochastic processes and random fields are studied from the point of view of
theirsecondorderpropertiesembodiedintwofunctions:themeanfunctionandthe
covariance function (or the covariance measure in the case of point process
randomfields).Thislimitedinformationsufficestosatisfymostdemandsofsignal
processingandhasmadethisclassofprocessesarichsourceofmodels:whiteand
coloured noises, ARMA series and, more recently, complex signals based on point
processes. Wide-sense stationary processes are, in addition to their usefulness,
supported by an elegant theory at the interface of Fourier analysis and Hilbert
spaces on one hand, and probability and stochastic processes on the other hand.
Thisbookpresentstheclassicalresultsinthefieldtogetherwithmorerecentones
on the power spectral measure (or Bartlett spectrum) of wide-sense stationary
point processes and related stochastic processes, such as (non-Poissonian) shot
noises, modulated point processes, random samples, etc.
The mathematical prerequisites on integration and probability theory are
reviewedintheAppendix,generallywithoutproof,exceptforthecompletenessof
Lp-spaces (stating in particular that the square-integrable functions form a Hilbert
space), because this result is the foundation of the second-order theory of
stochasticprocesses,togetherwithsomeresultsofHilbertspacetheory.Thelatter
are given, this time with proofs, in the course of the Chap. 1. No prerequisite on
stochastic processes is necessary, since the relevant models (especially Gaussian
processes and Brownian motion, but also Markov chains, Poisson processes and
renewal processes) are presented in detail in the main text as the need arises.
xi