Table Of ContentC O N T R OL A ND
D Y N A M IC S Y S T E MS
ADVANCES IN THEORY
AND APPLICATIONS
Edited by
C. T. LEONDES
School of Engineering and Applied Science
University of California, Los Angeles
Los Angeles, California
VOLUME 24: DECENTRALIZED/DISTRIBUTED
CONTROL AND DYNAMIC SYSTEMS
Part 3 of 3
1986
ACADEMIC PRESS, INC.
Harcourt Brace Jovanovich, Publishers
Orlando San Diego New York Austin
Boston London Sydney Tokyo Toronto
ACADEMIC PRESS RAPID MANUSCRIPT REPRODUCTION
COPYRIGHT © 1986 BY ACADEMIC PRESS, INC.
ALL RIGHTS RESERVED.
NO PART OF THIS PUBLICATION MAY BE REPRODUCED OR
TRANSMITTED IN ANY FORM OR BY ANY MEANS, ELECTRONIC
OR MECHANICAL, INCLUDING PHOTOCOPY, RECORDING, OR
ANY INFORMATION STORAGE AND RETRIEVAL SYSTEM, WITHOUT
PERMISSION IN WRITING FROM THE PUBLISHER.
ACADEMIC PRESS, INC.
Orlando, Florida 32887
United Kingdom Edition published by
ACADEMIC PRESS INC. (LONDON) LTD.
24-28 Oval Road, London NW1 7DX
LIBRARY OF CONGRESS CATALOG CARD NUMBER: 64-8027
ISBN 0-12-012724-5
PRINTED IN THE UNITED STATES OF AMERICA
86 87 88 89 9 8 7 6 5 4 3 2 1
P R E F A CE
In the series Control and Dynamic Systems this is the third volume of a trilogy
whose theme is advances in techniques for the analysis and synthesis of decentral-
ized or distributed control and dynamic systems. The subject of decentralized but
coordinated systems is emerging as a major issue in industrial and aerospace sys-
tems, and so this is an appropriately significant theme for this series at this time.
The three volumes of this trilogy will thus comprise the most comprehensive treat-
ment of the theory of this broad and complex subject and its many potential applica-
tions to date. It is in the various complex 4'real world" applications that many
practitioners may find these three volumes particularly useful. This includes the
articles on the many computational issues and techniques appearing in the textbook
literature for the first time.
The first article in this volume, "A Two-Level Parameter Estimation Algorithm
for Large-Scale Systems," by M. P. Spathopoulos, deals with the vital issue of
parameter estimation or determination in large-scale systems, which are rather
characteristic of decentralized systems. Some rather powerful new results are pre-
sented and verified by computer simulations which demonstrate that these new
techniques have the potential in diverse applications of producing considerable
savings in memory, computational effort, and in alleviating numerical inaccuracies.
There are a number of other significant advantages of the techniques in this article,
not the least of which is its suitability for multiprocessing systems. As in earlier
Control and Dynamic Systems, an essential design concept is that of suboptimal
control techniques and related suboptimal bounds. The next article by Sinai presents
a rather comprehensive treatment of this issue of central importance to a well-
developed theory of decentralized control. Substantial savings in computational
requirements result, greater insight into uncertainties associated with modeling
large-scale systems is gained, and numerous other advantages also result. The
article "Decentralized Control Using Observers," by Shahian, presents a rather
comprehensive treatment, with many new results of observers suitable for large-
scale decentralized systems. The next article, "System Zeros in the Decentralized
Control of Large-Scale Systems," by Kennedy, presents many new and rather
powerful results on the use of system zeros in the decentralized control of large-
scale systems, that decentralized transmission zeros are crucial to the stability of
composite closed-loop systems under a high-gain decentralized feedback. Funda-
mental relations to system controllability and observability are also developed.
Model reference adaptive control techniques have constituted an area of great in-
ix
χ PREFACE
terest in the past. What is essential if such techniques are to be developed and
applied to decentralized control systems is an extension of model reference adaptive
control to multiple-input multiple-output (ÌΙÌÏ) systems. The article "Direct
Model Reference Adaptive Control for a Class of ÌΙÌÏ Systems" by Sobel and
Kaufman provides the necessary and essential results. Finally, the last article of this
volume, "Passive Adaptation in Control System Design," by Sworder and Chou,
presents some simplified but highly effective and powerful control techniques. Such
a simplified but effective techniques approach is essential to a well-rounded theory
of the very broad and rather complex subject of decentralized but coordinated
control systems.
When the theme for this trilogy of volumes, of which this is the third and last,
was decided upon there seemed little doubt that it was most timely. However,
because of the substantially important contributions of the authors all volumes
promise to be not only timely but of substantial lasting value.
Control and Dynamic Systems
A Two-Level Parameter Estimation
Algorithm for Large-Scale Systems
M. P. SPATHOPOULOS
Department of Electrical Engineering
Imperial College of Science and Technology
London SW7 2BT, England
I. INTRODUCTION 1
II. LEAST SQUARE ESTIMATION—THE MULTIPLE
PROJECTION APPROACH 4
A. INTRODUCTION 4
B. LEAST SQUARE ESTIMATION OF ONE
RANDOM VECTOR IN TERMS OF ANOTHER 5
C. LEAST SQUARE ESTIMATION
OF GAUSSIAN RANDOM VECTORS 7
D. THE MULTIPLE PROJECTION APPROACH 11
E. EXAMPLE: STATE ESTIMATION 14
III. THE MULTIPLE PROJECTION ALGORITHM 16
A. PROBLEM FORMULATION , 16
B. DEVELOPMENT OF THE
NONRECURSIVE ALGORITHM 17
C. DEVELOPMENT OF THE
RECURSIVE ALGORITHM 31
IV. IMPLEMENTATION OF THE ALGORITHM 37
V. SIMULATION RESULTS 41
VI. CONCLUSIONS 55
REFERENCES 55
I. INTRODUCTION
System identification may be defined as the set of tech-
niques employed in building up mathematical models for real
processes. This is done by determining difference or differ-
ential equations, such that they describe the process in ac-
cordance with some predetermined criterion. The mathematical
1 Copyright © 1986 by Academic Press, Inc.
All rights of reproduction in any form reserved.
2 M. P. SPATHOPOULOS
models must be capable of representing the actual process (sys-
tem) behavior. The accuracy of identification can be measured
by the difference between the output of the real system and that
of the model. Usually system identification can be split into
two distinct phases: structure determination and parameter
estimation.
The structure of the model is usually determined from the
a priori physical knowledge that we have on the process. Param-
eter estimation is defined as the experimental determination of
values of parameters that govern the dynamic and/or nonlinear
behavior, assuming that the structure of the process model is
known. Parameter estimation usually requires experimental data
on the inputs and outputs of the system. It then enables us to
put precise values on the parameters so that the model describes
the real process and not a class of such processes. The ob-
served data or actual system behavior can be used for the de-
termination of unknown system parameters within the structure
of the model, which minimizes a given error criterion.
In large-scale systems the application of parameter estima-
tion techniques is a very difficult task, due to the high di-
mensionality inherent in the system. Several methods have been
proposed to deal with this difficulty, such as reducing the
order of the system, using perturbation techniques, and sensi-
tivity analysis [3]. Efforts have been devoted to decompose
large-scale problems into several smaller coupled subproblems.
Hierarchical system theory [7,11], which deals with system de-
composition and coordination, can be applied effectively to de-
couple these subproblems while at the same time allowing for
the coordination of their solutions to yield the original prob-
lem's solution.
TWO-LEVEL PARAMETER ESTIMATION ALGORITHM 3
Physical and conceptual hierarchical structures of large
systems may occur in different ways. The system may be com-
posed of interconnected subsystems with well-defined physical
boundaries. In other cases the system structure may be cate-
gorized according to a natural property such as time behavior
or may be characterized by its order of priority of different
parts of the system. Hierarchical system theory can then be
applied to these structures, where the physical and conceptual
structures are viewed basically the same.
Although the problem of optimization and control of large-
scale systems composed of interconnected dynamic subsystems has
previously been tackled from a deterministic point of view,
there is no general, well-established procedure developed for
stochastic problems such as the parameter estimation problem.
Arafeh and Sage [1] have considered this problem and have de-
veloped an interesting algorithm based on decomposition — coordi-
nation techniques. However, their algorithm is suboptimal and
it converges to the optimal solution only at the end of the ob-
servation period. An attempt using the maximum a posteriori
approach has been done in [8]. Hassan [4] applied a partitioning
approach for the optimal Kaiman filter for large-scale systems
in which, after a finite number of iterations between the sub-
systems and the coordinator, the optimum Kaiman estimator was
achieved. This method was limited to two subsystems only.
Hassan et at. [5] generalized the previous approach and devel-
oped a decentralized computational algorithm for the global
Kaiman filter using the multiple projection idea. The new fil-
ter used a hierarchical structure to perform successive ortho-
gonal! zations on the measured subspaces of each subsystem with-
in a two-level structure in order to provide the optimal estimate.
4 M. P. SPATHOPOULOS
This ensured substantial savings in computation time, stability,
and reduction of numerical inaccuracies. Thus, this idea has
proven to be an efficient technique for dealing with large-scale
interconnected dynamic systems. More recently Hassan et al.
[6] developed a new decentralized algorithm for the parameter
estimation problem by using the multiple projection approach
developed in [5]. This chapter gives a description of the al-
gorithm and proves that the algorithm gives the minimum variance
estimate after Ν iterations between the coordinator level and
the subsystems level, where Ν is the number of subsystems. It
develops the basic parameter estimation algorithm and then gen-
eralizes it to the recursive case. Simulation results of two
examples have indicated that this two-level algorithm provides
accurate estimates while requiring a modest computation effort.
II. LEAST SQUARE ESTIMATION—THE
MULTIPLE PROJECTION APPROACH
A. INTRODUCTION
Before the algorithm is derived, the basic principles of
least squares estimation are introduced and the multiple pro-
jection idea is analyzed.
In systems analysis, a fundamental problem is to provide
values for the unknown states or parameters of a system given
noisy measurements that are some functions of these states or
parameters. If we consider a certain number of measurements
{z^, z , z > which depend on a parameter θ, we can define
2 N
a function K (z z , z ) which will be called the estimate
N lf 2 N
of θ. Since the measurements z^ are, in general, random, the
estimate Κ(·) will also be a random variable. Since all func-
Ν
tions of z. could be estimates, the problem is to find an
TWO-LEVEL PARAMETER ESTIMATION ALGORITHM 5
estimate (which is a function of z^) that is optimal with re-
spect to some criterion. Also this estimate should possess
certain convergence properties with respect to the real value
of the parameters.
There are three principal estimators: (a) The maximum likeli-
hood estimator (MLE, (b) the Bayes1 estimator, and (c) the least
square estimator.
The MLE uses as its criterion the a priori conditional prob-
ability density function ρ(ζ |θ). The Bayes1 estimator uses as
Ν
its criterion the a posteriori probability density function
ρ(θ|ζ ). The criterion for the least mean-square estimator is
Ν
to minimize the mean-square estimation error. For the scalar
case, it is expressed as
min E{(θ - θ)2], (1)
where θ is the estimate of Θ.
In the case where θ is a vector, we minimize
E{ (θ - θ)Τ0(θ - θ) } (2)
where Q is a nonnegative definite symmetrical weighting matrix.
The least mean-square estimate is reviewed in the following
section.
B. LEAST SQUARE ESTIMATION
OF ONE RANDOM VECTOR
IN TERMS OF ANOTHER
Problem 1. Consider two jointly distributed random vectors
X and Y with respective dimensions η and m and with joint prob-
ability density function f (·, ·). Find the estimator X of
Λ, χ
X in terms of Y that is best in the sense that X minimizes
Ε { 11X - g(Y) ||2} over all functions g mapping R m into Rn.
6 M. P. SPATHOPOULOS
Proposition 1. The least square estimator X of X in terms
of Y in the sense of the above problem is the conditional ex-
pectation
X = E{X|Y} (3)
of X given Y, and the corresponding minimum mean-square error
2
is the conditional variance E{||x - E{X|Y}|| }.
Proof. It is known that
E{||X - g(Y) II 2} = Ε | ||Χ - g(Y) | | 2| Y }.
γ Εχ|γ
We would like to minimize
Ε | {ΧΤΧ - 2g(Y) TX + g(Y) Tg(Y)|Y}.
χ γ
This is written as
T T T
E{X X|Y} - 2g(Y) E{x|Y} + g(Y) g(Y)
2 2 2
= E{||g(Y) - E{X|Y}|| } + E{||X|| |Y} - ||E{X|Y}|| . (4)
The only term on the right-hand side of Eq. (4) involving g(Y)
is the first and this is uniquely minimized by setting
X = g(y) = E{X|Y}. (5)
It is easy to show that for any nonnegative matrix Q, X = E{X|Y}
Τ* τη η
minimizes E{ [X - g( Y )] Q [X - g(Y)]} over all functions g: R -> R .
In fact, if Q is positive definite the proof is unchanged if
2 TT Τ
||q|| is interpreted to mean q Qq and w q is replaced by w Qq.
If Q is nonnegative definite the same identification may be made
but ||q|| = I q ^ q l1^ 2 is in this case only a seminorm, and while
X = E{X|Y} minimizes the first term on the right-hand side of
Eq. (4), it does not do so uniquely.
Properties. The least square estimator is
(a) linear, that is,
E{AX + b|Y} = AE{X|Y} + b = AX + b (6)
f
