Nonlinear Filtering and Stochastic Control: Proceedings of the 3rd 1981 Session of the Centro Internazionale Matematico Estivo (C.I.M.E.), Held at Cortona, July 1–10, 1981

300 Pages·1982·3.18 MB·English

by  A. Bensoussan (auth.), Sanjoy K. Mitter, Antonio Moro (eds.)

Lecture Notes ni
Mathematics
Edited yb .A Dold and .B Eckmann
:seiresbuS enoizadnoF ,.E.M.I.C ezneriF
.resivdA otreboR itnoC
972
Nonlinear gniretliF
and Stochastic Control
Proceedings of the 3 dr 1891 Session of the
Centro I nternazionale Matematico Estivo ,).E.M.I.C(
Held at Cortona, July 1-10, 1891
Edited by .S .K Mitter and A. Moro
galreV-regnirpS
nilreB Heidelberg New kroY 1982
Editors:
Sanjoy K. Mitter
Department of Electrical Engineering and Computer Science and
Laboratory for Information and Decision Systems
Massachussetts Institute of Technology
Cambridge, MA 02139, USA
Antonio Moro
Istituto Matematico "U. Dini"
Universit& di Firenze, 50134 Firenze, Italy
AMS Subject Classifications (1980): 60 G 35, 60 H 10, 60 H 15, 93 E ,11
93 E 20, 93 E 25
ISBN 3-540-11976-0 Springer-Verlag Berlin Heidelberg New York
ISBN 0-387-11976-0 Springer-Verlag New York Heidelberg Berlin
This work is subject to copyright. All rights era ,devreser whether eht whole or part of the lairetam
is concerned, specifically those of translation, reprinting, esu-er of illustrations, ,gnitsacdaorb
reproduction yb photocopying machine or similar ,snaem dna storage in data .sknab Under
§ 54 of the namreG Copyright waL where copies era edam for other than private use, a eef si
elbayap to tfahcsllesegsgnutrewreV" Wort", Munich.
© yb galreV-regnirpS Berlin Heidelberg 3891
detnirP in ynamreG
Printing dna binding: Bettz Offsetdruck, .rtsgreB/hcabsmeH
012345-0413/6412
PREFACE
This volume is a record of the lectures and seminars presented at the C.I.M.E.
School on Nonlinear Filtering and Stochastic Control, held at Cortona, Italy, during
the period July l-July 10, 1981. The school concerned itself with recent developments
in stochastic partial differential equations, as well as theory and approximation
methods for non-linear filtering and stochastic control.
Many of the basic ideas of non-linear filtering and stochastic control were
developed during the sixties and early seventies. An account of these ideas can be
found in book form in Liptser-Shiryayev 41 , Kallianpur 31 , and Fleming-Rishel
13 . In non-linear filtering, the basic approach used in the books cited above was
the innovations approach combined with representation theorems for continuous and
discontinuous martingales as stochastic integrals (see the lectures of Grigelionis
and Kunita, this volume). The recent developments (post 1977) in non-linear filtering
however have been generally centred around the so-called Duncan-Mortensen-Zakai
equation, the basic equation for the evolution of the unnormalized conditional den-
sity. A preliminary account of these ideas can be found in the proceedings of the
Nato Advanced Study Institute, held in Les Arcs, June 1980. (cf. Hazewinkel-Willems
21). The present volume, as far as non-linear filtering is concerned, is a logica
successor to the Les Arcs meeting.
The following main themes were developed in the lectures and seminars given at
the school:
a) Development of General Nonlinear Filtering Equations using the Theory of Semimar-
tingales (Grigelionis, Kunita).
b) Forward and Backward Stochastic Differential Equations and the Diffeomorphism
Property of the flow corresponding to stochastic differential equations (Kunita).
c) Stochastic Partial Differential Equations and their relationship to Non-Linear
Filtering (Kunita, Mitter, Pardoux).
d) Existence and Uniqueness Theorems for the Zakai Equation (Kunita, Mitter, Pardoux).
IV
e) Smoothness of Densities and their relationship to Hypoellipticity (Kunita) .
f) Pathwise Nonlinear Filtering (Mitter, Pardoux).
g) Equations for Smoothing (Pardoux).
h) Relationship between Nonlinear Filtering and Stochastic Control (Mitter).
i) Geometrical Theory of Nonlinear Filtering (Mitter).
j) Variational Methods in Stochastic Control (Bensoussan).
k) Stochastic Control with Partial Observations (Pardoux).
i) Discretization of Stochastic Differential Equations (Talay).
m) Approximations for Nonlinear Filtering (Di Masi-Runggaldier).
n) Approximations for Stochastic Control (Bensoussan).
It is our hope that this volume will serve as a useful reference for research
workers in the field of nonlinear filtering and stochastic control.
It remains for us to thank the participants of the school in helping to create
a warm and stimulating atmosphere and the CIME Scientific Committee for their sup-
port in the organization of this school.
S. K. Mitter
A. Moro
References
1 W.H. Fleming - R.W. Rishel, "Deterministic and Stochastic Optimal Control",
Springer Verlag, 1975.
2 M. Hazewinkel - J.C. Willems, "Stochastic Systems: The Mathematics of Filtering
and Identification and Applications", Reidel Publishing Co., 1981.
3 G. Kallianpur, "Stochastic Filtering Theory", Springer Verlag, 1980.
4 R.S. Liptser - A.N. Shiryayev, "Statistics of Random Processes", Springer Verlag
1977.
C.I.M.E. Session on "Nonlinear Filterin~ and Stochastic Control"
List of Participants
P. Acquistapace, Scuola Normale Superiore, Piazza dei Cavalieri, 56100 Pisa
A. Adimurthi, Tata Institute, Bombay, India
C.A. de Carvalho Belo, Complexo ,I I.S.T., Av. Rovisco Pals, i000 Lisboa, Portugal
A. Bensoussan, INRIA, ~omaine de Voluceau-Rocquencourt, Le Chesnay, France
P. Berti, Istituto di Matematica Applicata "G. Sansone", Viale Morgagni 44, Firenze
C. Carraro, Laboratorio di Statistica, Univ. di Venezia, C~ Foscari, Venezia
F. Conti, Scuola Normale Superiore, Piazza dei Cavalieri, 56100 Pisa
G. Coppoletta, Via A. Omodeo 9, 56100 Pisa
C. Costantini, Istituto Matematico "G. Castelnuovo", P.le A. Moro 2, 00185 Roma
G. Del Grosso, Istituto Matematico "G. Castelnuovo", P.le A. Moro 2, 00185 Roma
G. Da Prato, Scuola Normale Superiore, Piazza dei Cavalieri, 56100 Pisa
G.B. Di Masi, CNR-LA.D.S.E.B., Corso Stati Uniti 4, 35100 Padova
J.M. Ferreira, Centro de Fisica da Mat~ria Condensada, Av. Prof. Gama Pinto 2,
1699 Lisboa, Portugal
A. Gerardi, Istituto Matematico "G. Castelnuovo", P.le A. Moro 2, 00185 Roma
S. Goldstein, Institute of Mathematics, University of Lodz, ul. Stefana Banacha 22,
90-238 Lodz, Poland
G. Goodman, Istituto Matematico "U. Dini", Viale Morgagni 67/A, 50134 Firenze
B. Grigelionis, Academy of Sciences, Lithuanian SSR, Institut of Mathematics
and Cybernetics, Vilnius, URSS
H. Kunita, Kyushu Univ., Faculty of Engineering, Dept. of Appl. Science,
Hakozaki, Fukuoka 812, Japan
F. Lambert, 75 rue Aublet, 13 300 Salon de Provence, France
J.M.N. Leitao, Centro de Analise e Processamento de Sinais, Complexo i, Instituto
Superior Tecnieo, Av. Rovisco Pais, i000 Lisboa, Portugal
W. Loges, Ruhruniversitat Bochum, Mathematisehes Institut, Geb. NA, Zimmer 3131,
463 Bochum, W. Germany
A. Lohmann, Girondelle 6, 463 Bochum, W. Germany
A. Lunardi, Scuola Normale Superiore, Piazza dei Cavalieri, 56100 Pisa
F. Marchetti, Istituto Matematico "G. Castelnuovo", P.le A. Moro 2, 00185 Roma
S.K. Mitter, Dept. of Electrical Engineering and Computer Science,
Massachusetts Institute of Technology, Cambridge, MA 02912, USA
G. Nappo, Istituto Matematico "G. Castelnuovo", P.le A. Moro 2, 00185 Roma
F. Pardoux, UER de Math&matique, Univ. de Provence, Marseille, France
IV
M. Pavon, LADSEB-CNR, Corso Stati Uniti a, 35100 Padova
M. Piccioni, Via G. Bitossi 34, 00100 Roma
G. Pistone, Via Tripoli 10/8, 10136 Torino
M. Pratelli, Via Di Mezzana 19, 56100 Pisa
S. Roelly, 116 Boulevard Saint Germain, 75006 Paris, France
W. Runggaldier, Seminario Matematico, Univ. di Padova, Via Belzoni 7, 35100 Padova
F. Spizzichino, Via Quinto F. Pittore 3, 00136 Roma
D. Talay, 15 rue Marengo, 13006 Marseille, France
B. Terreni, Istituto Matematieo "L. Tonelli", Via F. Buonarroti 2, 56100 Pisa
L. Tubaro, Via Brennero, 362, 38100 Trento
U. Viaro, Istituto di Elettrotecnica e di Elettronica, Via Gradenigo 6/A,
35100 Padova
P.A. Zanzotto, Via S. Antonio 7, 56100 Pisa
G. Zappa, Istituto di Informatiea e Sistemistiea, Facolt~ di Ingegneria,
Via S. Maria, 3, 50100 Firenze
CONTENTS
PREFACE By S. K. MITTER & A. MORO .............................................. III
LIST OF PARTICIPANTS ........................................................... V
I. Main Lectures
A. ,NASSUOSNEB Lectures no Stochastic Control
Part I - Variational Methods in Stochastic Control
Introduction .................................................... 1
i. Setting of the problem ....................................... I
2. Necessary conditions of optimality ........................... 8
3. Other forms of the stochastic maximum principles ............. 19
4. The case of control entering into the diffusion term ......... 25
5. Linear quadratic example ..................................... 33
References for Part I ........................................... 39
Part II - Discrete time stochastic control and approximation of continuous
time stochastic control problems
Introduction .................................................... 40
i. Review on the martingale formulation of controlled diffusions . 40
2. Semi-group approach to controlled diffusions ................. 45
3. Discrete time stochastic control problem ..................... 53
4. Identity of u and u ......................................... 58
References for Part II .......................................... 62
.B GRIGELIONIS, Stochastic noN Linear Filtering Equations dna Semimartingales
Introduction .................................................... 63
i. Preliminaries ................................................ 65
2. Stochastic non linear filtering equations .................... 73
3. Robustness in the theory of non linear filtering ............. 80
4. Densities of the conditional distributions of semimartingales . 90
References ...................................................... 98
.H KUNITA, Stochastic Partial Differential Equations connected with
Nonlinear Filtering
Part I - Stochastic Differential Equations
Introduction .................................................... I00
I. Stochastic Integrals ......................................... 104
2. It6 stochastic differential equation ......................... 108
3. It6's formula ................................................ 115
4. Stratonovich SDE ............................................. 119
5. Cauchy problem for parabolic differential equations .......... 123
IIIV
Part II - Stochastic Partial Differential Equations
Introduction .................................................... 127
i. Stochastic partial differential equations (I).Existence theorem 130
2. " " " " (II).Uniqueness theorem 136
3. Zakai equation for measure-valued process .................... 139
4. Backward SPDE ................................................ 142
5. Decomposition of solution .................................... 144
6. Hypoellipticity .............................................. 148
Appendix: Nonlinear filterin~ ............................................. 154
References ...................................................... 168
S. .K MITTER, Lectures no Nonlinear Filtering dna Stochastic Control
Chapter I. The Basic Equations of Nonlinear Filtering ..................... 170
" 2. On the Relation Between Nonlinear Filtering and Stochastic Control. 178
" 3. A Path Integral Calculation for a Class of Filtering Problems .. 188
" 4. Geometric Theory of Nonlinear Filtering ........................ 195
References ..................................................... 206
E. ,XUODRAP Equations of Nonlinear Filtering, dna Applications to Stochastic
Control with Partial Observation
Introduction ................................................... 208
Chapter I. The Reference Probability approach to the Nonlinear Filtering
Problem ...................................................... 209
" II. PDEs and Stochastic PDEs ..................................... 220
" III. Equations of nonlinear filtering, prediction and smoothing ... 229
" IV. Stochastic Control with Partial Observations ................. 240
References ..................................................... 246
II. Seminars
G.B. Di Masi, W.J. Runggaldier: On Approximation Methods for Nonlinear Filtering 249
B. Grigelionis, R. Mikulievicius: On Weak Convergence to Random Processes with
Boundary Conditions .......................... 260
D. Talay: How to Discretize Stochastic Differential Equations .................. 276
Announcements for 1982 Sessions ................................................ 293
LECTURES ON STOCLIASTIC CONTROL
A. BENSOUSSAN
University Paris-Dauphine and INRIA
PART I
VARIATIONAL SDOHTEM IN STOCHASTIC LORTNOC
,NOITCUDORTNI
We consider in this chapter the optimal control of diffusions. Our objective is
to derive the various forms of the Stochastic Maximum Principle° The initial work of
this area is due to H.J. Kushner 8. The most general treatment is that of J.M. Bismut
2, 3 where he allows random coefficients. U.G. Haussmann 6 has considered the
problem of necessary conditions among feedbacks. The methods used here are somewhat
different. They rely mainly on variational methods, which are very similar to those
used in the deterministic theory. We recover most of the results which exist in the
litterature, by more elementary methods. In particular the very general framework of
Bismut can b e avoided in most practical situations (I). We extend here the methods
and results given in A. Bensoussan I.
1, GNITTES FO EHT ,MELBORP
1.1. Notation. .snoitpmussA
Let g : R n X R m X o,T ~R n such that
(1.1) g is Borel measurable.
(1.2) Ig(=,v,t) - g(x',v,t)l ~ KIx-x'l
Ig(x,v,t) - g(x,v',t)l ~ Klv-v'l
Ig(x,v,t)l ~ ~1(1=1 + Ivl + 1)
)I( It remains that the stochastic convex analysis developped by Bismut is a very
powerful tool.
2
Let ~ : R n X o,T ~(Rn;R n) such that
(1.3) s is Borel
)t,x(sI - ~(x,,t)l ~ KI~-~'I
l)t,x(sI ~ ~i(~ + Ixl)
Let (~,~,P) be a probability space, and ~t be an increasing family of sub
s-algebras of ~, ~= oo . Let w(t) be an ~t standard Wiener process, with values in
R n (in particular w(t) is an ~t martingale).
Let
(1.4) Uad = non empty subset of R m.
eW note
)5.1(
a.e. v(O belongs to ~2(~,~t~p~Rm)l
which is a sub Hilbert space of L 2. We set
(1.6) ~ = Iv E ~(o,T) I v(t) E Uad , a.e., a.s. 1
and ~ is a convex closed subset of ~(o,T)
D
An element v of ~ will be called an admissible control° For any admissible control
we can solve the Ito equation
(1.7) dx = g(x(t),v(t),t)dt + ~(x(t),t)dw(t)
x(o) = x
o
where x is deterministic, to simplify.
o
Equation (1.7) has one and only one solution in L2(Q,~,P;C(o,T;Rn)). Moreover
vt, x(t) c L2(~,~t,p;Rn).
We say that x(t) is the state of the system.
We now define a cost functional as follows. Let
I( )8. ~(x,v,t) : R n X R m X o,T ~- R
be Borel, continuously differentiable with respect to (x,v), and