Path-dependent Hamilton-Jacobi-Bellman equations related to controlled stochastic functional differential systems

0.17 MB·English

by  Shaolin Ji

Path-dependent Hamilton-Jacobi-Bellman equations related to
controlled stochastic functional differential systems∗
Shaolin Ji† Shuzhen Yang‡
2 Abstract: In this paper, a stochastic optimal control problem is investigated in which the system
1
0 is governed by a stochastic functional differential equation. In the framework of functional Itô calculus,
2 we build the dynamic programming principle and the related Path-dependent Hamilton-Jacobi-Bellman

l
u (HJB) equation. We prove that the value function is the viscosity solution of the Path-dependent HJB
J
equation.

5

Keywords: stochastic functional differential equations; dynamic programming principle; viscosity
]
C
solution; Path-dependent HJB equations
O
.
h
1 Introduction
t
a
m
In many real world applications, the systems can only be modeled by stochastic systems whose evo-
[

lutions depend on the past history of the states. So in this paper, we study a stochastic optimal control
1
v problem in which the system is described by the following stochastic functional differential equation:
4
s s
9
1 Xγt,u(s)=γt(t)+Z b(Xrγt,u,u(r))dr+Z σ(Xrγt,u,u(r))dW(r).
1 t t
. The cost functional is
7
T
20 J(γt;u(·))=E[Z L(Xsγt,u,u(s))ds+Ψ(XTγt,u)].
t
1
For the initial datum γ ∈Λ, our optimal control problem is to find an admissible control u(·)∈ U[t,T]
: t
v
so as to minimize the cost functional J. In this case, the value function V :Λ→R is defined to be
i
X
r V(γt)= inf J(γt;u(·)).
a u(·)∈U[t,T]
It is well known that dynamic programming and related HJB equations is a powerful approach to
solving optimal control problems (see [7], [14] and [11]). Different from the HJB equations derived for
stochastic delay systems (see [1], [9] and [10]), we establish the dynamic programming principle and
derive the HJB equation in a new framework of functional Itô calculus. The functional Itô calculus and
path-dependent PDEs is recently introduced by Dupire [5] (for a recent account of this theory we refer
the reader to [2], [3] and [4]). Under the framework of functional Itô calculus, Peng and Wang [13]
obtained a nonlinear Feynman-Kac formula for classical solutions of path-dependent PDEs in terms of
non-MarkovianBSDEs. For the further development, the readers may refer to [12] and [6].
∗This work was supported by National Natural Science Foundation of China (No. 11171187, No. 10871118 and No.
10921101).
†InstituteforFinancialStudiesandInstituteofMathematics,ShandongUniversity,Jinan,Shandong250100,PRChina
([email protected],Fax: [phone]).
‡Schoolofmathematics, ShandongUniversity,Jinan,Shandong250100, PRChina. ([email protected]).
1
In our paper, we obtain the following path-dependent HJB equation
D V(γ )+ inf G(γ ,u,D V,D V)=0, γ ∈Λ,
t t t x xx t
u∈U
V(γ )=Ψ(γ ), γ ∈Λ,
T T T
where
1
G(γ ,u,p,P)= (Pσ(γ ,u)σ(γ ,u)T)+hp,b(γ )i+L(γ ,u),
t t t t t
2
∀(γ ,u,p,P)∈Λ×U ×Rn×Sn.
t
We prove that the value function is the viscosity solution of the Path-dependent HJB equation. In
addition, the stochastic verification theorem for the smooth case is also proved.
The paperis organizedasfollows: insection2,wepresentsomefundamentalresultsonfunctionalItô
calculus and formulate our stochastic optimal control problem. We establish the dynamic programming
principle in section 3. In section 4, we first derive the path-dependent HJB equation and prove the
stochasticverificationtheoremforthesmoothcase. Thenweprovethatthevaluefunctionistheviscosity
solution of the path-dependent HJB equation.
2 Problem formulation
2.1 Functional Itô calculus
We adopt the following notations in Dupire [5]. Let T >0 be fixed. For each t∈[0,T], we denote by
Λ the set of càdlàg Rd-valued functions on [0,t].
t
For each γ ∈ Λ the value of γ at time s ∈ [0,T] is denoted by γ(s). Thus γ = γ(s) is a
T 0≤s≤T
càdlàgfunction on [0,T]and its value attime s is γ(s). The path of γ up to time t is denoted by γ , i.e.,
t
γ = γ(s) ∈ Λ . We denote Λ = Λ . For each γ ∈ Λ and x ∈ Rd we denote by γ (s) the
t 0≤s≤t t t∈[0,T] t t t
value of γ at s∈[0,t] and γx :=(γ (s)S ,γ (t)+x) which is also an element in Λ .
t t t 0≤s<t t t
We now introduce a distance on Λ. Let h·,·i and |·| denote the inner product and norm in Rd. For
each 0≤t,t¯≤T and γt,γ¯t¯∈Λ, we denote
kγ k:= sup |γ (s)|,
t t
s∈[0,t]
kγt−γ¯t¯k:= sup |γt(s∧t)−γ¯t¯(s∧t¯)|,
s∈[0,t∨t¯]
d∞(γt,γ¯t¯):=sup0≤s≤t∨t¯|γt(s∧t)−γ¯t¯(s∧t¯)|+|t−t¯|.
It is obvious that Λ is a Banach space with respect to k·k. Since Λ is not a linear space, d is not
t ∞
a norm.
Definition 2.1. A functional u : Λ 7→ R is said to be Λ–continuous at γ ∈ Λ, if for any ε > 0 there
t
exists δ >0 such that for each γ¯t¯∈Λ with d∞(γt,γ¯t¯)<δ, we have |u(γt)−u(γ¯t¯)|<ε.
u is said to be Λ–continuous if it is Λ–continuous at each γ ∈Λ.
t
Definition 2.2. Let v :Λ7→R and γ ∈Λ be given. If there exists p∈Rd, such that
t
v(γx)=v(γ )+hp,xi+o(|x|) as x→0, x∈Rd.
t t
2
Then we say that v is (vertically) differentiable at γ and define D v(γ ) := p. v is said to be vertically
t x t
differentiable in Λ if D v(γ )exists foreach γ ∈Λ. Wecan similarly definetheHessian matrix D v(γ )
x t t xx t
which is an S(d)-valued functional defined on Λ, where S(d) is the set of all d×d symmetric matrices.
For each γ ∈Λ, we denote
t
γ (r)=γ (r∧t), r ∈[0,s].
t,s t
It is clear that γ ∈Λ .
t,s s
Definition 2.3. For a given γ ∈Λ, if we have
t
v(γ )=v(γ )+a(s−t)+o(|s−t|) as s→t, s≥t,
t,s t
then we say that v(γ ) is (horizontally) differentiable in t at γ and define D v(γ ) := a. u is said to be
t t t t
horizontally differentiable in Λ if D v(γ ) exists for each γ ∈Λ.
t t t
Definition2.4. Denoteby Cj,k(Λ) theset of functionals v :=(v(γ )) defined on Λ which are j times
t γt∈Λ
horizontally and k times vertically differentiable in Λ such that all these derivatives are Λ–continuous.
The following Itô’s formula was firstly obtained by Dupire [5] and then generalized by Cont and
Fournié [2].
Theorem 2.1 (Functional Itô’s formula). Let (Ω,F,(F(t)) ,P) be a probability space. If X is a
t∈[0,T]
continuous semi-martingale and u is in C1,2(Λ), then for any t∈[0,T),
t t
v(X )−v(X )= D v(X )ds+ D v(X )dX(s)
t 0 Z s s Z x s
0 0
1 t
+ D v(X )dhXi(s) P −a.s.
2Z xx s
0
2.2 Stochastic optimal control problem
LetW be ad-dimensionalstandardBrownianmotiondefinedonacomplete filteredprobabilityspace
(Ω,F,P;{F(t)} ), where {F(t)} is the P-augmentation of the natural filtration generated by the
t≥0 t≥0
Brownian motion W.
LetT >0begiven. Forany(t,γ )∈[0,T]×Λ ,considerthefollowingcontrolledstochasticfunctional
t t
differential equation:
dXγt,u(s)=b(Xγt,u,u(s))ds+σ(Xγt,u,u(s))dW(s), s∈[t,T], (2.1)
s s
with the initial condition Xγt,u =γ , where u(·)={u(s),s∈[t,T]} is a control process taking values in
t t
acompactsetU ofanEuclideanspaceand b:Λ×U →Rn, σ :Λ×U →Rn×Rd aregivendeterministic
functions.
The definition of strong solutions of (2.1) and the admissible controls are given as follows.
Definition 2.5. Given the d-dimensional standard Brownian motion {W(s),s ∈ [0,T]} and the control
process {u(s),s∈[t,T]}, a process {Xγt,u(s)}s∈[t,T] is said to be a strong solution of (2.1) on the interval
[t,T] and through the initial datum (t,γ )∈[0,T]×Λ if it satisfies the following conditions:
t t
1. Xγt,u =γ ;
t t
2. Xγt,u(s) is F(s)-measurable for each s∈[t,T];
3. The process {Xγt,u(s),s ∈ [t,T]} is continuous and it satisfies the following stochastic integral
equation P −a.s.
s s
Xγt,u(s)=γt(t)+Z b(Xrγt,u,u(r))dr+Z σ(Xrγt,u,u(r))dW(r).
t t
3
Definition 2.6. For each t ∈ [0,T], a five-tuples α = (Ω,F,P,W(·),u(·)) is said to be an admissible
control if it satisfies the following conditions:
1. (Ω,F,P) is a complete probability space.
2. W ={W(s),s∈[t,T]} is a d-dimensional standard Brownian motion on (Ω,F,P) over [t,T] with
W(t)=0 a.s. and F(s,t)=σ{W(τ),t≤τ ≤s} augmented by all the P-null sets in F.
3. u:[t,T]×Ω→U is an F(s,t),s∈[t,T]-adapted process on (Ω,F,P).
4. Under the control process u(·) = {u(s),s ∈ [t,T]}, equation (2.1) admits a unique strong solution
Xγt,u(·) = {Xγt,u(s),s ∈ [t,T]} defined on (Ω,F,P;{F(s,t),s ∈ [t,T]}) and through each initial datum
γ ∈Λ .
t t
5. The control process u(·) satisfies
T
E[ |L(Xγt,u,u(s))|ds+|Ψ(Xγt,u)|]<∞,
Z s T
t
where L:Λ×U →R and Ψ:Λ→R represent the running and terminal cost functions, respectively.
The collection of admissible controls α = (Ω,F,P,W(·),u(·)) over the time interval [t,T] is denoted
by U[t,T]. We will write u(·)∈U[t,T] or α=(Ω,F,P,W(·),u(·)) interchangeably, whenever there is no
danger of ambiguity.
Givenanyinitialdatumγ ∈Λandanyadmissiblecontrolu(·)∈U[t,T],wedefinethecostfunctional
t
T
J(γt;u(·))=E[Z L(Xsγt,u,u(s))ds+Ψ(XTγt,u)].
t
Then our optimal control problem can be stated as follows:
For each γ ∈Λ, find a u(·) ∈U[t,T] so as to minimize the cost functional J. In this case, the value
t
function V :Λ→R is defined to be
V(γ )= inf J(γ ;u(·)). (2.2)
t t
u(·)∈U[t,T]
WeassumebandσareuniformlycontinuousandtheysatisfythefollowinglineargrowthandLispschitz
conditions.
Assumption 2.1. There exists a constant c>0 such that
|b(x1,u)−b(x2,u)|+|σ(x1,u)−σ(x2,u)|≤ckx1−x2 k,
t t t t t t
∀(x1,u),(x2,u)∈Λ×U.
t t
Assumption 2.2. There exists a constant K >0 such that
|b(Φ ,u)|+|σ(Φ ,u)|≤K(1+kΦ k), ∀(Φ ,u)∈Λ×U.
t t t t
Let L and Ψ be two uniformly continuous real-valued functionals on Λ×U and Λ, respectively.
Assumption 2.3. There exist a constant c >0 and integer k >0 such that
1 1
|L(Φ ,u)|≤c (1+kΦ kk1), |Ψ(Φ )|≤c (1+kΦ kk1) (2.3)
t 1 t T 1 T
for all (Φ ,u)∈Λ×U and Φ ∈Λ.
t T
4
Assumption 2.4. There exists a constant c>0 such that
|L(Φ1,u)−L(Φ2,u)|≤ckΦ1−Φ2 k,
t t t t
|Ψ(Φ1)−Ψ(Φ2)|≤ckΦ1 −Φ2 k,
T T T T
∀(Φ1,u),(Φ2,u)∈Λ×U.
T T
Remark. Let Assumptions 2.1 and 2.2 hold. Then there exists a unique solution X for equation
(2.1) (see [8]).
3 Dynamic programming principle
In this section, we build dynamic programming principle for our optimal control problem.
We first give some lemmas.
Denote the set of all continuous functions ϕ:[0,T]→Rn by C([0,T];Rn).
Lemma 3.1. (Theorem 2.10 of [14]) Let (Ω,F,P) be a complete probability space and (U,d) a Polish
space. Letξ :[0,T]×Ω→Rn beacontinuousprocessandFξ =σ(ξ(s):0≤s≤t). Then ϕ:[0,T]×Ω→
t
U is {Fξ} -adapted if and only if there exists an η ∈An(U) such that ∀t∈[0,T]
t t≥0 T
ϕ(t,ω)=η(t,ξ(·∧t,ω)), P −a.s. ω
where An(U) is the set of all {B (C([0,T];Rn))}-progressively measurable processes.
T t+
Lemma 3.2. Let γ ∈ Λ and (Ω,F,P,W(·),u(·)) ∈ U[t,T]. Then for any s ∈ [t,T] and F(r,t)
t t≤r≤s
progressively measurable process γˆ,
J(γˆs(ω);u(·))=E{ sT L(Xrγˆs,u,u(r))dr+Ψ(XTγˆs,u)|F(s,t)}(ω), P −a.s. ω. (3.1)
R
Proof. Sinceu(·)is{F(r,t)} -adapted,byLemma3.1thereexistsafunctionalψ ∈An(U)such
t≤r≤T T
that ∀r∈[t,T]
u(r,ω)=ψ(r,W(·∧r,ω)), P −a.s. ω.
For s ∈ [t,T], let γˆ be a F(r,t) progressively measurable process. We consider the following
t≤r≤s
equation:
dXγˆs,ψ(r)=b(Xγˆs,ψ,ψ(r,W(·∧r)))dr+σ(Xγˆs,ψ,ψ(r,W(·∧r)))dW(r),
r r
(3.2)
X =γˆ , r ∈[s,T],
s s
Due to Assumption 2.1 and 2.2, there exists a unique solution of (3.2).
Note that
P{ω¯ :γˆ (ω¯)=γˆ (ω)|F(s,t)}(ω)
s s
= E{I |F(s,t)}(ω)
{γˆs(ω¯)=γˆs(ω)}
= I (ω)
{γˆs(ω¯)=γˆs(ω)}
= 1, P −a.s. ω.
5
Thenthereexistsan Ω ∈F withP(Ω )=1suchthatforanyfixedω ∈Ω ,γˆ becomesadeterministic
0 0 0 0 s
function γˆ (ω ) under the new probability space (Ω,F,P(· | F(s,t)(ω )). On the other hand, for any
s 0 0
r≥s,
u(r,ω)=ψ(r,W(·∧r))=ψ(r,W(·∧r,ω)+W(s,ω)),
f
where W(r) = W(r)−W(s) is a standard Brownian motion. Thus, u(r) is adapted to W(r) for r ≥ s
under tfhe probability measure P(·|F(s,t))(ω0). We obtain that f
(Ω,F,P(·|F(s,t))(ω ),W(·),u(·))∈U[t,T]
0
f
under the new probability space (Ω,F,P(· | F(s,t))(ω )). By the uniqueness of (3.2), we obtain our
0
result. (cid:3)
The value function (2.2) has the following properties.
Lemma3.3. ThevaluefunctionV is continuousandthereexistsaconstantc >0andapositiveinteger
2
k such that, for every γ ∈Λ,
2 t
|V(γ )|≤c (1+kγ k)k2.
t 2 t
In addition, ∀ε>0, there exists a constant δ >0 such that |V(φ )−V(ϕ )|<ε whenever kφ −ϕ k<δ.
t s t s
Proof. Due to Assumption 2.3, it is easy to prove that V has at most polynomial growth. We only
give a brief proof of the continuity property.
Fix u(·)∈U[0,T] and the initial datum φ and ϕ , respectively. For 0≤t≤s≤T, we have
t s
J(φ ;u(·))−J(ϕ ;u(·))
t s
= E[ sL(Xφt,u,u(r))dr+ T[L(Xφt,u,u(r))−L(Xϕs,u,u(r))]dr+Ψ(Xφt,u)−Ψ(Xϕs,u)].
t r s r r T T
R R
By Assumption 2.3 and 2.4, there exists a constant c>0 such that
|J(φ ;u(·))−J(ϕ ;u(·))|≤c(|s−t|(1+E[kXφt,u k])+E[kXφt,u−Xϕs,u k]).
t s T T T
Then, by Assumptions 2.1 and 2.2, we can obtain the basic estimations of solution of (2.1) which leads
to the following result:
There exists a constant δ >0 such that
|J(φ ;u(·))−J(ϕ ;u(·))|≤ε,
t s
whenever kφ −ϕ k<δ.
t s
Thus, it yields that
|V(φ )−V(ϕ )|≤ε.
t s
This completes the proof. (cid:3)
The following dynamic programming principle holds.
Theorem 3.1. Let Assumption 2.1−2.4 hold. Then for any γ ∈Λ and any s∈[t,T],
t
V(γt)= inf E[ tsL(Xrγt,u,u(r))dr+V(Xsγt,u)]. (3.3)
u(·)∈U[t,s] R
6
Proof. Denote the rightsideof(3.3)byV¯(γ ). Foranyε>0,thereexistsanu(·)∈U[t,T]suchthat
t
V(γ )+ε>J(γ ;u(·))
t t
= E{ T L(Xγt,u,u(r))dr+Ψ(Xγt,u)}
t r T
R
= E{ sL(Xγt,u,u(r))dr+E[ T L(Xγt,u,u(r))dr+Ψ(Xγt,u)|F(s,t)]}
t r s r T
R R
= E{ tsL(Xrγt,u,u(r))dr+E[ sT L(XrXsγt,u,u,u(r))dr+Ψ(XTXsγt,u,u)|F(s,t)]}
R R
= E{ sL(Xγt,u,u(r))dr+J(Xγt,u)}
t r s
R
≥ E{ tsL(Xrγt,u,u(r))dr+V(Xsγt,u)}≥V¯(γt).
R
Conversely, by Lemma 3.3, for any ε>0, there exists a δ =δ(ε) such that whenever kγ −γˆ k<δ,
t t
|J(γ ;u(·))−J(γˆ;u(·))|+|V(γ )−V(γˆ)|≤ε, ∀u(·)∈U[t,T].
t t t t
Let{D } beaBorelpartitionofC([0,s];Rn)withdiameterdiam(D )<δ. Choosexj ∈D . Foreach
j j≥1 j s j
j, there is (Ω ,F ,P ,W (·),u (·))∈U[s,T] such that
j j j j j
J(xj;u (·))≤V(xj)+ε.
s j s
Hence for any x ∈D , we can get
s j
J(x ;u (·))≤J(xj;u (·))+ε≤V(xj)+2ε≤V(x )+3ε. (3.4)
s j s j s s
By the definition of the admissible control, there is a functional ψ ∈An(U) such that
j T
u (r,ω)=ψ (r,W (·∧r,ω)), P −a.s. ω ∈Ω , r∈[s,T].
j j j j j
Now for any(Ω,F,P,W(·),u(·))∈U[t,T],let X(·)=Xγt,u(·) denote the correspondingstate trajectory.
Define a new control
u˜(r,ω)=u(r,ω), if r ∈[t,s),
u˜(r,ω)=ψ (r,W(·∧r,ω)−W(s,ω)), if r ∈[s,T], X (ω)∈D .
j r j
Clearly, u˜(·)∈U[t,T]. Thus,
V(γ ) ≤J(γ ;u˜(·))
t t
=E{ T L(Xγt,u˜,u˜(r))dr+Ψ(Xγt,u˜)}
t r T
R
=E{ sL(Xγt,u,u(r))dr+E[ T L(Xγt,u˜,u˜(r))dr+Ψ(Xγt,u˜)|F(s,t)]}
t r s r T
R R
=E{ tsL(Xrγt,u,u(r))dr+E[ sT L(XrXsγt,u˜,u˜,u˜(r))dr+Ψ(XTXsγt,u˜,u˜)|F(s,t)]}
R R
=E{ sL(Xγt,u,u(r))dr+J(Xγt,u˜)}
t r s
R
≤E{ sL(Xγt,u,u(r))dr+V(Xγt,u)+3ε},
t r s
R
where the last inequality is due to (3.4). Taking the infimum over u(·) ∈ U[t,T], we obtain the result.
(cid:3)
7
4 Path-dependent HJB equation
4.1 Classical solution
In this subsection, we suppose that the value function V is smooth, i.e., V ∈C1,2(Λ).
Theorem 4.1. Let Assumption 2.1−2.4 hold and the value function V ∈C1,2(Λ). Then V is a solution
of the following path-dependent second-order partial differential equation:
D V(γ )+ infG(γ ,u,D V,D V)=0, γ ∈Λ,
t t t x xx t
u∈U (4.1)
V(γ )=Ψ(γ ), γ ∈Λ,
T T T
where
G(γ ,u,p,P)= 1tr[Pσ(γ ,u)σ(γ ,u)T]+hp,b(γ ,u)i+L(γ ,u),
t 2 t t t t
∀(γ ,u,p,P)∈Λ×U ×Rn×Sn (Sn is the set of all n order symmetric matrix).
t
Proof. Fixγt ∈Λandu∈U. LetXγt,u(·)bethe statetrajectorycorrespondingtothe initialdatum
γ and the control u(r)≡u. By Theorem 3.1 and Functional Itô’s formula, we obtain
t
0 ≤ E[V(Xsγt,u)−V(γt)] + 1 E[ sL(Xγt,u,u)dt]
s−t s−t t r
R
= s−1tE[ ts{Dt(γt)+G(Xrγt,u,u,DxV(Xrγt,u),DxxV(Xrγt,u))}dt],
R
Let s↓t, we have
0≤D V(γ )+G(γ ,u,D V(γ ),D V(γ )).
t t t x t xx t
This result in
0≤D V(γ )+ infG(γ ,u,D V(γ ),D V(γ )). (4.2)
t t t x t xx t
u∈U
Ontheotherhand,foranyε>0,0≤t<s≤T withs−t>0smallenough,thereexistsau (·)∈U[t,T]
ε,s
such that
s
V(γt)+ε(s−t)≥E[Z L(Xrγt,uε,s,uε,s(r))dt+V(Xsγt,uε,s)].
t
By Functional Itô’s formula,
ε ≥ E[V(Xsγt,uε,s)−V(γt)] + 1 E[ sL(Xγt,uε,s,u (r))dt]
s−t s−t t r ε,s
R
= 1 E[ s{D V(γ )+G(Xγt,uε,s,u (r),D V(Xγt,uε,s),D V(Xγt,uε,s))}dt]
s−t t t t r ε,s x r xx r
R
≥ 1 E[ s{D V(γ )+ infG(Xγt,uε,s,u,D V(Xγt,uε,s),D V(Xγt,uε,s))}dt],
s−t t t t u∈U r x r xx r
R
Let s↓t, we have
ε≥D V(γ )+ infG(γ ,u,D V(γ ),D V(γ )). (4.3)
t t t x t xx t
u∈U
Combining (4.2) and (4.3), we obtain our conclusion. (cid:3)
8
4.1.1 Classical stochastic verification theorem
Now we give the classical stochatic verification theorem.
Theorem 4.2. Let Assumption 2.1−2.4 hold. V ∈C1,2(Λ) is a solution of the HJB equation (4.1).
Then
V(γ )≤J(γ ;u(·)),∀u(·)∈U[t,T], γ ∈Λ. (4.4)
t t t
Futhermore, an admissible pair (X¯(·),u¯(·)) is optimal if and only if
D V(X¯ ) =−infG(X¯ ,u,D V(X¯ ),D V(X¯ ))
t s s x s xx s
u∈U (4.5)
=−G(X¯ ,u¯(s),D V(X¯ ),D V(X¯ )),
s x s xx s
a.e. s∈[t,T], P −a.s.
Proof. Applying functional Itô’s formula to V(Xγt,u), we obtain
s
V(γt)= E[Ψ(XTγt,u)]−E[ tT{DtV(Xrγt,u)+hDxV(Xrγt,u),b(Xrγt,u,u(r))i
R
+21tr(DxxV(Xrγt,u)σ(Xrγt,u,u(r))σ(Xrγt,u,u(r))T)}dt]
= J(γt;u(·))−E[ tT L(Xrγt,u,u(r))dt]−E[ tT{DtV(Xrγt,u)
R R (4.6)
+hDxV(Xrγt,u),b(Xrγt,u,u(r))i+ 12tr(DxxV(Xrγt,u)σ(Xrγt,u,u(r))σ(Xrγt,u,u(r))T)}dt]
≤ J(γt;u(·))−E[ tT{DtV(Xrγt,u)+infu∈UG(Xrγt,u,u,DxV(Xrγt,u),DxxV(Xrγt,u))}dt]
R
= J(γ ;u(·)).
t
Then (4.4) follows.
Next, applying (4.6) to (X¯(·),u¯(·)), we have
T
V(γ )=J(γ ;u¯(·))−E[ [D V(X¯ )+G(X¯ ,u¯(s),D V(X¯ ),D V(X¯ ))]ds].
t t Z t s s x s xx s
t
If (4.5) holds, then
V(γ )=J(γ ;u¯(·))
t t
and (X¯(·),u¯(·)) is optimal.
Conversely, if (X¯(·),u¯(·)) is optimal, then
V(γ )=J(γ ;u¯(·)),
t t
Applying Itô’s formula to V(X¯ ), we have
s
T
V(γ )=J(γ ;u¯(·))−E[ [D V(X¯ )+G(X¯ ,u¯(s),D V(X¯ ),D V(X¯ ))]ds].
t t Z t s s x s xx s
t
Note that
D V(X¯ )+G(X¯ ,u¯(s),D V(X¯ ),D V(X¯ ))≤0.
t s s x s xx s
We have immediately
D V(X¯ )+G(X¯ ,u¯(s),D V(X¯ ),D V(X¯ ))=0.
t s s x s xx s
Thus, we get the desired result. (cid:3)
9
4.2 Viscosity solution
In this section, under Assumption 2.1−2.4,we show that the value function V is actually a viscosity
solution of the path-dependent HJB equation (4.1). We first give the definition of the viscosity solution
of (4.1) as follows.
Definition 4.1. Let w ∈C(Λ), we say that w is a viscosity subsolution of (4.1), if ∀δ >0, ∀Γ∈C1,2(Λ),
γ ∈Λ satisfying Γ≥w on Λ and Γ(γ )=w(γ ), we have
t 0≤r≤δ t+r t t
S
D Γ(γ )+ inf G(γ ,u,D Γ,D Γ)≥0.
t t t x xx
u∈U
where
1
G(γ ,u,p,P)= tr[Pσ(γ ,u)σ(γ ,u)T]+hp,b(γ ,u)i+L(γ ,u),
t t t t t
2
∀(γ ,u,p,P)∈Λ×U ×Rn×Sn.
t
We say that w is a viscosity supersolution of (4.1), if ∀δ > 0, ∀Γ ∈C1,2(Λ), γ ∈Λ satisfying Γ≤ w on
t
Λ and Γ(γ )=w(γ ), we have
0≤r≤δ t+r t t
S
D Γ(γ )+ inf G(γ ,u,D Γ,D Γ)≤0.
t t t x xx
u∈U
We say that w is a viscosity solution of (4.1) if it is both a viscosity supersolution and a viscosity
subsolution of (4.1).
Then we have the following main result:
Theorem 4.3. Let Assumption 2.1−2.4 hold. Then the value function V is a viscosity solution of the
path-dependent equation (4.1).
Proof. (1) For any δ > 0 and Γ ∈ C1,2(Λ), let Γ ≥ V on Λ and Γ(γ ) = V(γ ). Fix a
0≤r≤δ t+r t t
u∈U. Let Xγt,u(·) be the solution of(2.1) with the control u(s)S≡u. Then by Functional Itô’s formula,
for t≤s≤t+δ, we have
E[Γ(Xsγt,u)]−Γ(γt)]
(4.7)
= E[ ts(DtΓ(Xrγt,u)+G(γt,u,DxΓ(Xrγt,u),DxxΓ(Xrγt,u))−L(Xrγt,u,u))dr].
R
On the other hand, for any s∈[t,t+δ], by the dynamic programming principle, we have
s
V(γt)≤E[Z L(Xrγt,u,u)dr+V(Xsγt,u)].
t
Notice that Γ≥V and Γ(γ )=V(γ ). By (4.7), we get
t t
0≤ E[ tsL(Xrγt,u,u)dr+V(Xsγt,u)]−V(γt)
R
≤ E[ tsL(Xrγt,u,u)dr+Γ(Xsγt,u)]−Γ(γt)
R
≤ E[ tsDtΓ(Xrγt,u)+G(γt,u,DxΓ(Xrγt,u),DxxΓ(Xrγt,u))dr],
R
Dividing both sides by (s−t), we have
0≤ s−1tE[ tsDtΓ(Xrγt,u)+G(γt,u,DxΓ(Xrγt,u),DxxΓ(Xrγt,u))dr].
R
10