Stochastic Approximation and Its Application

369 Pages·2002·6.34 MB·English

by  Han-Fu Chen

Stochastic Approximation and Its Applications
Nonconvex Optimization and Its Applications
Volume 64
Managing Editor:
Panos Pardalos
Advisory Board:
J.R. Birge
Northwestern University, U.S.A.
Ding-Zhu Du
University of Minnesota, U.S.A.
C. A. Floudas
Princeton University, U.S.A.
J. Mockus
Lithuanian Academy of Sciences, Lithuania
H. D. Sherali
Virginia Polytechnic Institute and State University, U.S.A.
G. Stavroulakis
Technical University Braunschweig, Germany
The titles published in this series are listed at the end of this volume.
Stochastic Approximation
and Its Applications
by
Han-Fu Chen
Institute of Systems Science,
Academy of Mathematics and System Science,
Chinese Academy of Sciences,
Beijing, P.R. China
KLUWER ACADEMIC PUBLISHERS
NEW YORK,BOSTON, DORDRECHT, LONDON, MOSCOW
eBookISBN: 0-306-48166-9
Print ISBN: 1-4020-0806-6
©2003 Kluwer Academic Publishers
NewYork, Boston, Dordrecht, London, Moscow
Print ©2002 Kluwer Academic Publishers
Dordrecht
All rights reserved
No part of this eBook maybe reproducedor transmitted inanyform or byanymeans,electronic,
mechanical, recording, or otherwise, without written consent from the Publisher
Created in the United States of America
Visit Kluwer Online at: http://kluweronline.com
and Kluwer's eBookstore at: http://ebooks.kluweronline.com
Contents
Preface ix
Acknowledgments xv
1. ROBBINS-MONRO ALGORITHM 1
1.1 Finding Zeros of a Function. 2
1.2 Probabilistic Method 4
1.3 ODE Method 10
1.4 Truncated RM Algorithm and TS Method 16
1.5 Weak Convergence Method 21
1.6 Notes and References 23
2. STOCHASTIC APPROXIMATION ALGORITHMS WITH
EXPANDING TRUNCATIONS 25
2.1 Motivation 26
2.2 General Convergence Theorems by TS Method 28
2.3 Convergence Under State-Independent Conditions 41
2.4 Necessity of Noise Condition 45
2.5 Non-Additive Noise 49
2.6 Connection Between Trajectory Convergence and Property
of Limit Points 57
2.7 Robustness of StochasticApproximation Algorithms 67
2.8 Dynamic Stochastic Approximation 82
2.9 Notes and References 93
3. ASYMPTOTIC PROPERTIES OF STOCHASTIC
APPROXIMATION ALGORITHMS 95
3.1 Convergence Rate: Nondegenerate Case 96
3.2 Convergence Rate: Degenerate Case 103
3.3 Asymptotic Normality 113
v
vi STOCHASTIC APPROXIMATION AND ITS APPLICATIONS
3.4 AsymptoticEfficiency 130
3.5 Notes and References 149
4. OPTIMIZATION BY STOCHASTIC APPROXIMATION 151
4.1 Kiefer-Wolfowitz Algorithm with Randomized Differences 153
4.2 Asymptotic Properties of KW Algorithm 166
4.3 Global Optimization 172
4.4 Asymptotic Behavior of Global Optimization Algorithm 194
4.5 Application to Model Reduction 210
4.6 Notes and References 218
5. APPLICATION TO SIGNAL PROCESSING 219
5.1 Recursive Blind Identification 220
5.2 Principal Component Analysis 238
5.3 Recursive Blind Identification by PCA 246
5.4 Constrained Adaptive Filtering 265
5.5 Adaptive Filtering by Sign Algorithms 273
5.6 Asynchronous Stochastic Approximation 278
5.7 Notes and References 288
6. APPLICATION TO SYSTEMS AND CONTROL 289
6.1 Application to Identification and Adaptive Control 290
6.2 Application to Adaptive Stabilization 305
6.3 Application to Pole Assignment for Systems with Unknown
Coefficients 316
6.4 Application to Adaptive Regulation 321
6.5 Notes and References 327
Appendices 329
329
A.1 Probability Space 329
A.2 Random Variable and Distribution Function 330
A.3 Expectation 330
A.4 Convergence Theorems and Inequalities 331
A.5 Conditional Expectation 332
A.6 Independence 333
A.7 Ergodicity 333
335
B.1 Convergence Theorems for Martingale 335
B.2 Convergence Theorems for MDS I 339
B.3 Borel-Cantelli-Lévy Lemma 340
Contents vii
B.4 Convergence Criteria for Adapted Sequences 341
B.5 Convergence Theorems for MDS II 343
B.6 Weighted Sum of MDS 344
References 347
Index 355
Preface
Estimating unknown parameters based on observation data contain-
ing information about the parameters is ubiquitous in diverse areas of
both theory and application. For example, in system identification the
unknown system coefficients are estimated on the basis of input-output
data of the control system; in adaptive control systems the adaptive
control gain should be defined based on observation data in such a way
that the gain asymptotically tends to the optimal one; in blind chan-
nel identification the channel coefficients are estimated using the output
data obtained at the receiver; in signal processing the optimal weighting
matrix is estimated on the basis of observations; in pattern classifica-
tion the parameters specifying the partition hyperplane are searched by
learning, and more examples may be added to this list.
All these parameter estimation problems can be transformed to a
root-seeking problem for an unknown function. To see this, let de-
note the observation at time i.e., the information available about the
unknown parameters at time It can be assumed that the parameter
under estimation denoted by is a root of some unknown function
This is not a restriction, because, for example, may
serve as such a function. Let be the estimate for at time Then
the available information at time can formally be written as
where
Therefore, by considering as an observation on at with
observation error the problem has been reduced to seeking the
root of based on
It is clear that for each problem to specify is of crucial importance.
The parameter estimation problem is possible to be solved only if
ix
x STOCHASTIC APPROXIMATION AND ITS APPLICATIONS
is appropriately selected so that the observation error meets the
requirements figured in convergence theorems.
If and its gradient can be observed without error at any desired
values, then numerical methods such as Newton-Raphson method among
others can be applied to solving the problem. However, this kind of
methodscannot be used here, because in addition to the obvious problem
concerning the existence and availability of the gradient, the observations
are corrupted by errors which may contain not only the purely random
component but also the structural error caused by inadequacy of the
selected
Aiming at solving the stated problem, Robbins and Monro proposed
the following recursivealgorithm
to approximate the sought-for root where is the step size. This
algorithm is now called the Robbins-Monro (RM) algorithm. Follow-
ing this pioneer work of stochastic approximation, there have been a
large amount of applications to practical problems and research works
on theoretical issues.
At beginning, the probabilistic method was the main tool in con-
vergence analysis for stochastic approximation algorithms, and rather
restrictive conditions were imposed on both and For example,
it is required that the growth rate of is not faster than linear as
tends to infinity and is a martingale difference sequence [78].
Though the linear growth rate condition is restrictive, as shown by sim-
ulation it can hardly be simply removed without violating convergence
for RM algorithms.
To weaken the noise conditions guaranteeing convergence of the algo-
rithm, the ODE (ordinary differential equation) method was introduced
in [72, 73] and further developed in [65]. Since the conditions on noise
required by the ODE method may be satisfied by a large class of
including both random and structural errors, the ODE method has been
widely applied for convergence analysis in different areas. However, in
this approach one has to a priori assume that the sequence of estimates
is bounded. It is hard to say that the boundedness assumption is
more desirable than a growth rate restriction on
The stochastic approximation algorithm with expanding truncations
was introduced in [27], and the analysis method has then been improved
in [14]. In fact, this is an RM algorithm truncated at expanding bounds,
and for its convergence the growth rate restriction on is not re-
quired. The convergence analysis method for the proposed algorithm
is called the trajectory-subsequence (TS) method, because the analysis