Random Signals Estimation and Identification: Analysis and Applications

636 Pages·1986·9.57 MB·English

by  Nirode Mohanty (auth.)

RANDOM SIGNALS
ESTIMATION
AND IDENTIFICATION
ANALYSIS AND APPLICATIONS
Nirode Mohanty
The Aerospace Corporation
Los Angeles, California
Van Nostrand Reinhold Electrical/Computer Science and Engineering Series
~ VAN NOSTRAND REINHOLD COMPANY
'~ New York _____
Copyright © 1986 by Van Nostrand Reinhold Company Inc.
Softcover reprint of the hardcover I st edition 1986
Library of Congress Catalog Card Number: 85-12973
[SBN-13: 978-94-011-7043-7 e-ISBN-13: 978-94-011-7041-3
001: 10.100/978-94-011-7041-3
All rights reserved. No part of this work covered by the copyright hereon may
be reproduced or used in any form or by any means-graphic, electronic, or
mechanical, including photocopying, recording, taping, or information storage
and retrieval systems-without permission of the publisher.
Manufactured in the United States of America
Published by Van Nostrand Reinhold Company Inc.
115 Fifth Avenue
New York, New York 10003
Van Nostrand Reinhold Company Limited
Molly Millars Lane
Wokingham, Berkshire RG11 2PY, England
Van Nostrand Reinhold
480 Latrobe Street
Melbourne, Victoria 3000, Australia
Macmillan of Canada
Division of Gage Publishing Limited
164 Commander Boulevard
Agincourt, Ontario MIS 3C7, Canada
15 14 13 1211 10 987654321
Library of Congress Cataloging in Publication Data
Mohanty, Nirode.
Random signals estimation and identification.
(Van Nostrand Reinhold electrical/computer science
and engineering series)
Bibliography: p.
Includes index.
1. Statistical communication theory. 2. Random noise
theory. 3. Estimation theory. 1. Title. II. Series.
TK5102.5.M64 1986 621.38'0436 85-12973
ISBN-13: 978-94-011-7043-7
To
Dr. Richard Ernest Bellman,
Professor of Mathematics, Electrical Engineering
and Medicine
University of Southern California
Van Nostrand Reinhold
Electrical/Computer Science and Engineering Series
Sanjit Mitra -Series Editor
HANDBOOK OF ELECTRONIC DESIGN AND ANALYSIS PROCEDURES USING
PROGRAMMABLE CALCULATORS, by Bruce K. Murdock
COMPILER DESIGN AND CONSTRUCTION, by Arthur B. Pyster
SINUSOIDAL ANALYSIS AND MODELING OF WEAKLY NONLINEAR CIRCUITS,
by Donald D. Weiner and John F. Spina
APPLIED MULTIDIMENSIONAL SYSTEMS THEORY, by N. K. Bose
MICROWAVE SEMICONDUCTOR ENGINEERING, by Joseph F. White
INTRODUCTION TO QUARTZ CRYSTAL UNIT DESIGN, by Virgil E. Bottom
DIGITAL IMAGE PROCESSING, by William B. Green
SOFTWARE TESTING TECHNIQUES, by Boris Beizer
LIGHT TRANSMISSION OPTICS, Second edition, by Dietrich Marcuse
REAL TIME COMPUTING, edited by Duncan Mellichamp
HARDWARE AND SOFTWARE CONCEPTS IN VLSI, edited by Guy Rabbat
MODELING AND IDENTIFICATION OF DYNAMIC SYSTEMS, by N. K. Sinha and
B. Kuszta
COMPUTER METHODS FOR CIRCUIT ANALYSIS AND DESIGN, by Jiri Vlach and
Kishore Singhal
HANDBOOK OF SOFTWARE ENGINEERING, edited by C. R. Vick and C. V.
Ramamoorthy
SWITCHED CAPACITOR CIRCUITS, by Phillip E. Allen and Edgar Sanchez
Sinencio
SOFTWARE SYSTEM TESTING AND QUALITY ASSURANCE, by Boris Beizer
MODERN DC-TO-DC SWITCHMODE POWER CONVERTER CIRCUITS, by Rudolf
P. Severns and Gordon E. Bloom
DIGITAL COMMUNICATIONS, by Israel Korn
RANDOM SIGNALS ESTIMATION AND IDENTIFICATION, by Nirode Mohanty
Preface
The techniques used for the extraction of information from received or ob
served signals are applicable in many diverse areas such as radar, sonar,
communications, geophysics, remote sensing, acoustics, meteorology, med
ical imaging systems, and electronics warfare. The received signal is usually
disturbed by thermal, electrical, atmospheric, channel, or intentional inter
ferences. The received signal cannot be predicted deterministically, so that
statistical methods are needed to describe the signal. In general, therefore,
any received signal is analyzed as a random signal or process.
The purpose of this book is to provide an elementary introduction to
random signal analysis, estimation, filtering, and identification. The emphasis
of the book is on the computational aspects as well as presentation of com
mon analytical tools for systems involving random signals. The book covers
random processes, stationary signals, spectral analysis, estimation, optimiz
ation, detection, spectrum estimation, prediction, filtering, and identification.
The book is addressed to practicing engineers and scientists. It can be used as
a text for courses in the areas of random processes, estimation theory, and
system identification by undergraduates and graduate students in engineer
ing and science with some background in probability and linear algebra.
Part of the book has been used by the author while teaching at State
University of New York at Buffalo and California State University at Long
Beach. Some of the algorithms presented in this book have been successfully
applied to industrial projects.
Random signal processes are dealt with in Chapter 1, with emphasis on
Gaussian, Brownian, Poisson, and Markov processes. Mean square calculus
and renewal theory are briefly discussed.
Chapter 2 is devoted to stationary random processes along with spectral
analysis, narrow-band processes, the Karhunen-Loeve expansion, entropy,
zero crossing detectors, and nonlinear systems with random inputs.
A comprehensive account of estimation theory is given in Chapter 3.
Maximum-likelihood estimation, mean square estimation, maximum a priori
estimation, the Cramer-Rao bound, and interval estimation are covered.
Optimum filters including Wiener filtering are discussed for white and
colored noise. Elements of signal detection are included in this chapter.
Spectral estimation methods including the periodogram, autoregressive,
maximum entropy, maximum likelihood, Pisarenko, and Prony methods are
vii
viii PREFACE
discussed in Chapter 4. Adaptive spectral density estimation and cross
spectral estimation are briefly described.
Chapter 5 deals with prediction, filtering, and identification. Kalman filter
ing, extended Kalman filtering, and recursive identification algorithms are
given in this chapter.
Each chapter contains a set of worked-out problems, exercises, and biblio
graphic notes for further study.
Appendix 1 contains a brief survey oflinear systems analysis, Z transforms,
sampling theory, matrices, and orthogonal transforms. Elementary proba
bility, random variables and distribution theory are treated in Appendix 2.
Appendix 3 deals with the stochastic integral. Elements of Hilbert space are
discussed in Appendix 4.
The book concludes with a detailed bibliography and reference list. Many
of the publications listed therein are suitable for further study.
The author would like to thank the authors of the books and papers cited
in the bibliography, which have greatly helped to write this book. He would
also thank his friends and the management at The Aerospace Corporation
for their encouragement and support. Readers are invited to send their com
ments and corrections which will be appreciated very much.
Los Angeles, California. Nirode Mohanty
Notations
Capital letters x, Y, Z etc. denote random variables or matrices. However, N
is used also as a number of samples. Bold letters X, x, A, a etc. indicate
vectors or matrices. x, y, z etc. stand for real numbers or dummy variables.
P[X ~ x] = Probability of r.v. X is less than equal to x.
X = (X1,XZ' ••• 'Xn)
(n dimensional vector)
Fx(x) = Probability distribution function of r.v. X, x is a real number
~ F(X)
fx(x) = Probability density function of r.v. X, x is a real number
~ f(X)
fx(xls) = Conditional probability density function of random vector X, x
is a vector quantity given s
~ f(Xls)
E(X) = Expected value of r.v. X, E is the expected operator
Var(X) = Variance ofr.v. X.
X(f) = ~(x(t))
~ Fourier transform of x(t)
X*(f) = Complex conjugate of X(f)
X(s) = 2(x(t))
~ Laplace transform of x(t)
X(z) ~ Z transform of {Xd or {X(k)}
A -1 = Inverse of matrix A
X' = Transpose of vector X or matrix X
ix
Contents
Preface / vii
1 Random Signals / 1
1.0 Introduction / 1
1.1 Characterization and Classification / 1
1.2 Correlation and Covariance Functions / 10
1.3 Gaussian Processes and Wiener Processes / 18
1.4 Poisson Process / 24
1.5 Mean Square Calculus / 34
1.6 Markov Process / 40
1.7 Renewal Process / 86
1.8 Bibliographical Notes / 98
Exercises / 98
2 Stationary Random Signals / 109
2.1 Introduction / 109
2.2 Linear Systems with Random Signal Input / 115
2.3 Cross Covariance and Coherence / 126
2.4 Narrowband Noise Process / 137
2.5 Orthogonal Expansion and Sampling / 141
2.6 Ergodicity and Entropy / 155
2.7 Zero Crossing Detectors / 170
2.8 Nonlinear Systems / 180
2.9 Bibliographical Notes / 197
Exercises / 197
3 Estimation, Optimization, and Detection / 212
3.0 Introduction / 212
3.1 Sampling Distribution / 212
3.2 Estimation of Parameter: Point Estimation / 217
3.3 Estimation Criteria / 223
3.4 Maximum Likelihood Estimation / 228
3.5 Linear Mean Square Estimation / 235
3.6 Method of Least Squares: Regression Models / 238
xi
xii CONTENTS
3.7 Interval Estimation: Confidence Interval / 247
3.8 Cramer-Rao Inequality / 249
3.9 Estimation in Colored Noise / 259
3.10 Optimum Linear Filters / 266
3.11 Signal Detection / 292
3.12 Bibliographical Notes / 309
Exercises / 309
4 Spectral Analysis / 319
4.0 Introduction / 319
4.1 The Periodogram Approach / 319
4.2 Spectral Windows / 329
4.3 Autoregressive Method / 337
4.4 The Maximum Entropy Method / 343
4.5 Maximum Likelihood Estimator / 354
4.6 Pisarenko and Prony Methods / 359
4.7 Adaptive Lattices Method / 369
4.8 Cross Spectral Estimation / 387
4.9 Bibliographical Notes / 395
Exercises / 396
5 Prediction, Filtering, and Identification / 412
5.0 Introduction / 412
5.1 State Space Representation / 413
5.2 The Innovation Process / 457
5.3 Linear Prediction and Kalman Filtering / 476
5.4 Smoothing / 498
5.5 Extended Kalman Filtering / 504
5.6 System Identification / 509
5.7 Bibliographical Notes / 526
Exercises / 527
Appendix 1. Linear Systems Analysis / 549
Appendix 2. Probability / 566
Appendix 3. Stochastic Integrals / 589
Appendix 4. Hilbert Space / 598
Bibliography / 609
Index / 619