Properties of viscosity solutions for

stochastic control problems

Speaker: Giacomo Vizzari
September 1st , 2023
Outline

1 A brief introduction to stochastic control

2 Viscosity solutions in PDEs

3 Viscosity solutions in stochastic control

2 of 38
Introduction

We briefly recall some notions of stochastic calculus which will

prove useful next. We define:
 Z t  
2 2
Mloc := Xs progressively measurable : E |Xs | ds < ∞ ∀t .
0

A process Xt ∈ M2loc is an Itô process if dXs = Ψs ds + Φs dBs for

some coefficients Ψs ∈ M1loc and Φs ∈ M2loc .

3 of 38
Introduction

We briefly recall some notions of stochastic calculus which will

prove useful next. We define:
 Z t  
2 2
Mloc := Xs progressively measurable : E |Xs | ds < ∞ ∀t .
0

A process Xt ∈ M2loc is an Itô process if dXs = Ψs ds + Φs dBs for

some coefficients Ψs ∈ M1loc and Φs ∈ M2loc .
A classical stochastic control problem is governed by a Stochastic
Differential Equation (SDE) on (Ω, F, (F)t≥0 , P):

dXs = b(s, Xs , αs )ds + σ(s, Xs , αs )dBs .

3 of 38
SDEs for Stochastic Control

dXs = b(s, Xs , αs )ds + σ(s, Xs , αs )dBs

B is a (n-dimensional) Brownian motion;

αs = α(s, Xs ) is a measurable control on the filtered space;
b(t, x, a) and σ(t, x, a) are both Lipschitz in x, uniformly w.r.
to (t, a).

4 of 38
SDEs for Stochastic Control

dXs = b(s, Xs , αs )ds + σ(s, Xs , αs )dBs

B is a (n-dimensional) Brownian motion;

αs = α(s, Xs ) is a measurable control on the filtered space;
b(t, x, a) and σ(t, x, a) are both Lipschitz in x, uniformly w.r.
to (t, a).
Since Xt is an Itô process, for any φ ∈ C 2 (R+ × Rn ) we can apply
Itô’s formula:
 
1 t 2
dφ = ∂t φ + b · ∂x φ + tr(σσ Dx φ) ds + (σ · ∂x φ)dBs .
2

We notice that the last term is bounded, hence it is a martingale

when integrated on [0, t).
4 of 38
Stochastic control problem
We will only consider the finite horizon problem in the following
pages, but similar definitions and results can be found for
T = +∞. We fix a finite horizon 0 < T < ∞.

5 of 38
Stochastic control problem
We will only consider the finite horizon problem in the following
pages, but similar definitions and results can be found for
T = +∞. We fix a finite horizon 0 < T < ∞.

Let A be the set of control processes α such that

b(0, αs ), σ(0, αs ) ∈ M2 ([0, T )). By Itô’s theory we get a unique
a.s. continuous solution Xst,x , given initial conditions (t, x), for
each fixed α. We can then define the gain function:
Z T 
t,x t,x
J(t, x, α) := E f (s, Xs , αs )ds + g (XT ) .
t

We would like to find the optimal gain function, or value function:

v (t, x) := sup J(t, x, α).

α∈A(t,x)

5 of 38
Assumptions on f and g
For the previous problem to make sense we need some assumptions
on the measurable functions g and f .

We assume g to either be lower-bounded or satisfy a quadratic

growth condition |g (x)| ≤ C (1 + |x|2 ). We also restrict ourselves
to the set of controls A(t, x) ⊂ A such that if α ∈ A(t, x) then:
Z T 
E f (s, Xst,x , αs )ds <∞
t

6 of 38
Assumptions on f and g
For the previous problem to make sense we need some assumptions
on the measurable functions g and f .

We assume g to either be lower-bounded or satisfy a quadratic

growth condition |g (x)| ≤ C (1 + |x|2 ). We also restrict ourselves
to the set of controls A(t, x) ⊂ A such that if α ∈ A(t, x) then:
Z T 
E f (s, Xst,x , αs )ds <∞
t

When f satisfies a quadratic growth assumption in x, or in other

words there exists C > 0, k : A → R+ such that

|f (t, x, a)| ≤ C (1 + |x|2 ) + k(a)

for all (t, x, a) then the constant controls α ≡ a ∈ A are all in

A(t, x).
6 of 38
DPP and HJB
We now recall the most important results for stochastic control
problems:
Dynamic Programming Principle (DPP)
Let (t, x) ∈ [0, T ) × Rn , then for any stopping time θ:
Z θ 
v (t, x) = sup E f (s, Xst,x , αs )ds + v (θ, Xθt,x )) .
α∈A(t,x) t

7 of 38
DPP and HJB
We now recall the most important results for stochastic control
problems:
Dynamic Programming Principle (DPP)
Let (t, x) ∈ [0, T ) × Rn , then for any stopping time θ:
Z θ 
v (t, x) = sup E f (s, Xst,x , αs )ds + v (θ, Xθt,x )) .
α∈A(t,x) t

Hamilton-Jacobi-Bellman equation (HJB)

Let v (t, x) be a C 2 value function. Then it satisfies:

−∂t v (t, x) − H(t, x, Dx v (t, x), Dx2 v (t, x)) = 0, v (T , x) = g (x).

7 of 38
Verification theorem

In the HJB equation we used H to indicate our system’s

Hamiltonian:  
1 t
H(t, x, p, M) = sup b(x, a) · p + tr(σσ (x, a)M) + f (t, x, a)
a∈A 2

8 of 38
Verification theorem

In the HJB equation we used H to indicate our system’s

Hamiltonian:  
1 t
H(t, x, p, M) = sup b(x, a) · p + tr(σσ (x, a)M) + f (t, x, a)
a∈A 2

Verification theorem
Let w ∈ C 1,2 ([0, T ) × Rn ) ∩ C 0 ([0, T ] × Rn ) with quadratic
growth be a supersolution of the HJB equation, then w ≥ v .
Also if we assume w (T , x) = g (x) and there exists α̂ such that H
attains a maximum in α̂ and its associated process X̂st,x is such
that α̂(s, X̂st,x ) ∈ A(t, x). Then w = v .

So C 2 solutions of HJB correspond with our value functions under

certain conditions. What about value functions without C 2
regularity?
8 of 38
Viscosity solutions (1)

We consider a non-linear second order partial differential equation:

(E ) F (x, u(x), Du(x), D 2 u(x)) = 0 for t ∈ O

Let’s assume the following ellipticity condition to hold for all

(x, r , p) ∈ O × R × Rd , where O ⊂ Rd is a bounded domain:

F (x, r , p, A) ≤ F (x, r , p, B) whenever A ≥ B.

A function u : O → R is a classical supersolution(/subsolution) of

(E ) if u ∈ C 2 (O) and

F (x, u(x), Du(x), D 2 u(x)) ≥ (/ ≤)0 for x ∈ O.

9 of 38
Viscosity solutions (2)

Let’s define the semicontinuous envelopes of u as:

u∗ (x) = lim′ inf u(x ′ ) u ∗ (x) = lim sup u(x ′ )

x →x x ′ →x

10 of 38
Viscosity solutions (2)

Let’s define the semicontinuous envelopes of u as:

u∗ (x) = lim′ inf u(x ′ ) u ∗ (x) = lim sup u(x ′ )

x →x x ′ →x

A function u is a viscosity supersolution of (E ) if

F (x0 , u∗ (x0 ), Dφ(x0 ), D 2 φ(x0 )) ≥ 0

for all x0 ∈ O, φ ∈ C 2 (O) such that x0 minimizes (u∗ − φ) on O.

10 of 38
Viscosity solutions (2)

Let’s define the semicontinuous envelopes of u as:

u∗ (x) = lim′ inf u(x ′ ) u ∗ (x) = lim sup u(x ′ )

x →x x ′ →x

A function u is a viscosity supersolution of (E ) if

F (x0 , u∗ (x0 ), Dφ(x0 ), D 2 φ(x0 )) ≥ 0

for all x0 ∈ O, φ ∈ C 2 (O) such that x0 minimizes (u∗ − φ) on O.

A function u is a viscosity subsolution of (E ) if

F (x0 , u ∗ (x0 ), Dφ(x0 ), D 2 φ(x0 )) ≤ 0

for all x0 ∈ O, φ ∈ C 2 (O) such that x0 maximizes (u ∗ − φ) on O.

10 of 38
Viscosity solutions (3)

A function u is a viscosity solution of (E ) if it’s both a viscosity

supersolution and subsolution.

11 of 38
Viscosity solutions (3)

A function u is a viscosity solution of (E ) if it’s both a viscosity

supersolution and subsolution.

The concept of viscosity derives from the idea that the same
conditions hold for classical solutions because of the ellipticity
assumptions: classical solutions are viscosity solutions.

11 of 38
Viscosity solutions - Stability (1)
Viscosity solutions require a lesser degree of regularity to be
defined, but we have no straightforward uniqueness result
compared to the classical case. We will now try to study the
properties of viscosity solutions starting with their stability.
Proposition (Stability):
Let uε be a lower semicontinuous viscosity supersolution of the
equation Fε (x, uε (x), Duε (x), D 2 uε (x)) = 0 for x ∈ O, with
(Fε )ε>0 sequence of continuous elliptic functions. Suppose that
(ε, x) → uε (x) and (ε, z) → Fε (z) are locally bounded, and define

u(x) := lim inf uε (x ′ ), F (z) := lim sup Fε (z ′ ).

(ε,x ′ )→(0,x) (ε,z ′ )→(0,z)

Then, u is a (lower semicontinuous) viscosity supersolution of the

equation F (x, u(x), Du(x), D 2 u(x)) = 0 for x ∈ O.

12 of 38
Viscosity solutions - Stability (2)

Proof. Let φ ∈ C 2 (O) such that there exists x̄ strict minimizer of

u − φ. By hypothesis there exists a sequence (εn , xn ) → (0, x̄) such
that uεn (xn ) → u(x̄). Let’s choose xn such that xn ∈ B = Br (x̄).

13 of 38
Viscosity solutions - Stability (2)

Proof. Let φ ∈ C 2 (O) such that there exists x̄ strict minimizer of

u − φ. By hypothesis there exists a sequence (εn , xn ) → (0, x̄) such
that uεn (xn ) → u(x̄). Let’s choose xn such that xn ∈ B = Br (x̄).

Now let x̄n be local minimizer of uεn − φ on B and claim:

x̄n −−−→ x̄ and uεn (x̄n ) −−−→ u(x̄).

n→∞ n→∞

If this holds we deduce that x̄n ∈ B for large n, so x̄n is a local

minimizer of uεn − φ and:

Fεn (x̄n , uεn (x̄n ), Dφ(x̄n ), D 2 φ(x̄n )) ≥ 0

and by definition of F , taking a lim sup we have proved our thesis.

It remains to prove that our claim is true.

13 of 38
Viscosity solutions - Stability (3)

We now prove our previous claim. We know by compactness that

x̄n → x̃ up to a subsequence for some x̃ ∈ B. We then have that,
since x̄n is a minimizer of uεn − φ and by definition of u:

(u − φ)(x̄) = lim (uεn − φ)(xn )

n→∞
≥ lim sup(uεn − φ)(x̄n )
n→∞
≥ lim inf (uεn − φ)(x̄n )
n→∞
≥ (u − φ)(x̃).

We can conclude by using the fact that x̄ is a strict minimizer, so

x̃ = x̄ and our claim holds.

14 of 38
Viscosity solutions - Definition by semijets
Let q(x, r , p, M) := r + p · x + 12 Ax · x, x ∈ Rd .

15 of 38
Viscosity solutions - Definition by semijets
Let q(x, r , p, M) := r + p · x + 12 Ax · x, x ∈ Rd .

If v ∈ LSC (O), and (x0 , φ) ∈ O × C 2 (O) be such that x0 is a local

minimizer of the difference (v − φ) in O, then, with p := Dφ(x0 )
and A := D 2 φ(x0 ), we have from a second order Taylor expansion
that:

v (x) ≥ v (x0 ) + φ(x) − φ(x0 ) = q(x − x0 , v (x0 ), p, A) + o(|x − x0 |2 ).

15 of 38
Viscosity solutions - Definition by semijets
Let q(x, r , p, M) := r + p · x + 12 Ax · x, x ∈ Rd .

If v ∈ LSC (O), and (x0 , φ) ∈ O × C 2 (O) be such that x0 is a local

minimizer of the difference (v − φ) in O, then, with p := Dφ(x0 )
and A := D 2 φ(x0 ), we have from a second order Taylor expansion
that:

v (x) ≥ v (x0 ) + φ(x) − φ(x0 ) = q(x − x0 , v (x0 ), p, A) + o(|x − x0 |2 ).

We can now define the subjet set of v at x0 in O:


−
JO v (x0 ) := (p, A) ∈ Rd × Sd :

v (x) ≥ q(x − x0 , v (x0 ), p, A) + o(|x − x0 |2 ) .

15 of 38
A fundamental result from this definition is that a function
v ∈ LSC (O) is a viscosity supersolution to (E ) if and only if:
−
F (x, v (x), p, A) ≥ 0 for all (p, A) ∈ JO v (x) for all x ∈ O.

One side of the statement is trivial, but the other implication

−
requires to construct an appropriate φ for each (p, A) ∈ JO v (x).

16 of 38
A fundamental result from this definition is that a function
v ∈ LSC (O) is a viscosity supersolution to (E ) if and only if:
−
F (x, v (x), p, A) ≥ 0 for all (p, A) ∈ JO v (x) for all x ∈ O.

One side of the statement is trivial, but the other implication

−
requires to construct an appropriate φ for each (p, A) ∈ JO v (x).

We can define the superjet the same way for u ∈ USC (O):

+
JO u(x0 ) := (p, A) ∈ Rd × Sd :

2
v (x) ≤ q(x − x0 , u(x0 ), p, A) + o(|x − x0 | ) ,

and a similar statement to the one above can be formulated for

viscosity subsolutions.

16 of 38
Closure of semijets

±
J¯O w (x) := (p, A) ∈ Rd × Sd : (xn , w (xn ), pn , An ) → (x, w (x), p, A),

±
for some xn ∈ O, (pn , An ) ∈ JO w (xn ) .

Proposition
If F is upper-semicontinuous, then v ∈ LSC (O) is a viscosity
supersolution if and only if:
−
F (x, v (x), p, A) ≥ 0 for all (p, A) ∈ J¯O v (x) for all x ∈ O.

If F is lower-semicontinuous, then u ∈ USC (O) is a viscosity

subsolution if and only if:
F (x, u(x), p, A) ≤ 0 for all (p, A) ∈ J¯O
+
u(x) for all x ∈ O.

17 of 38
Crandall-Ishii’s Lemma
Lemma: Crandall-Ishii
Let O ⊂ Rd be open and locally compact, u ∈ USC (O) and
2
v ∈ LSC (O). We assume for some (x0 , y0 ) ∈ O2 , φ ∈ C 2 (O )
that:
(u − v − φ)(x0 , y0 ) = max(u − v − φ)
O2

Then, for each ε > 0, there exist A, B ∈ Sn such that

−
(Dx φ(x0 , y0 ), A) ∈ J¯O
+
u(x0 ), (−Dy φ(x0 , y0 ), B) ∈ J¯O v (y0 ),

and the following inequality holds

 in thesense of symmetric
A 0
matrices in Sn2 , with ΩA,−B = :
0 −B

−(ε−1 + D 2 φ(x0 , y0 ) )I2n ≤ ΩA,−B ≤ D 2 φ(x0 , y0 ) + εD 2 φ(x0 , y0 )2 .

18 of 38
Crandall-Ishii - consequences

We will only consider the specific case with φ(x, y ) = α2 |x − y |2 ,

α = ε−1 .

In this scenario for α → +∞ (and ε → 0) we have that

(u(x) − v (y ) − φ(x, y )) can only be maximized for x, y close to
each other; in particular for α big enough we can expect that the
maximum is reached for x = y and we only need to maximize
u − v on O.

19 of 38
Crandall-Ishii - consequences

We will only consider the specific case with φ(x, y ) = α2 |x − y |2 ,

α = ε−1 .

In this scenario for α → +∞ (and ε → 0) we have that

(u(x) − v (y ) − φ(x, y )) can only be maximized for x, y close to
each other; in particular for α big enough we can expect that the
maximum is reached for x = y and we only need to maximize
u − v on O.

From the lemma we get:

−
(1) (α(x0 − y0 ), A) ∈ J¯O
+
u(x0 ), (α(x0 − y0 ), B) ∈ J¯O v (y0 ),
   
In 0 In −In
(2) − 3α ≤ ΩA,−B ≤ 3α .
0 In −In In

19 of 38
Comparison principle - assumptions

Before we formulate the comparison principle for viscosity solutions

it is necessary to add some further assumption on F . Suppose
that:
(strict monotonicity ) There exists γ > 0 such that

F (x, r , p, A) − F (x, r ′ , p, A) ≥ γ(r − r ′ )

for all r ≥ r ′ , (x, p, A) ∈ O × Rd × Sd .

20 of 38
Comparison principle - assumptions

Before we formulate the comparison principle for viscosity solutions

it is necessary to add some further assumption on F . Suppose
that:
(strict monotonicity ) There exists γ > 0 such that

F (x, r , p, A) − F (x, r ′ , p, A) ≥ γ(r − r ′ )

for all r ≥ r ′ , (x, p, A) ∈ O × Rd × Sd .

(growth assumption) There is a function π : R+ → R+ with
π(0+) = 0, such that

F (y , r , α(x−y ), B)−F (x, r , α(x−y ), A) ≤ π(α|x−y |2 +|x−y |)

for all x, y ∈ O, r ∈ R and A, B satisfying (2).

20 of 38
Comparison principle for viscosity solutions

We can finally prove a comparison principle for viscosity solutions.

Theorem: Comparison principle
Let O ⊂ Rd be open bounded, let F be an elliptic operator
satisfying the above assumptions. Let u ∈ USC (O) and
v ∈ LSC (O) be viscosity subsolution and supersolution of the
equation (E ), respectively. Then:

u|∂O ≤ v |∂O =⇒ u ≤ v on O.

This principle gives us conditions for which we have uniqueness for

viscosity solutions given boundary conditions. Furthermore, let w
be a viscosity solution. Then w ∗ and w∗ are viscosity subsolution
and supersolution respectively and w∗ ≥ w ∗ ≥ w ≥ w∗ .

21 of 38
Comparison principle proof (1)

By contradiction, assume that δ = (u − v )(z) > 0 for some z ∈ O.

22 of 38
Comparison principle proof (1)

By contradiction, assume that δ = (u − v )(z) > 0 for some z ∈ O.

Step 1. O is compact and u − v is upper-semicontinuous, so:
n α o α
Mα := sup u(x) − v (y ) − |x − y |2 = u(xα )−v (yα )− |xα − yα |2
O2 2 2

for some (xα , yα ). By compactness there exists a subsequence

(xn , yn ) := (xαn , yαn ), with αn → +∞, converging to some
2
(x̂, ŷ ) ∈ O .

22 of 38
Comparison principle proof (1)

By contradiction, assume that δ = (u − v )(z) > 0 for some z ∈ O.

Step 1. O is compact and u − v is upper-semicontinuous, so:
n α o α
Mα := sup u(x) − v (y ) − |x − y |2 = u(xα )−v (yα )− |xα − yα |2
O2 2 2

for some (xα , yα ). By compactness there exists a subsequence

(xn , yn ) := (xαn , yαn ), with αn → +∞, converging to some
2
(x̂, ŷ ) ∈ O . If we have that (proof in step 4):

(3) x̂ = ŷ , αn |xn − yn |2 → 0, Mαn → (u − v )(x̂) = sup(u − v )

O

then δ ≤ Mαn = u(xn ) − v (yn ) − α2n |xn − yn |2 , by definition of Mαn ,

and, since u ≤ v on ∂O, this gives us (xn , yn ) ∈ O eventually.

22 of 38
Comparison principle proof (2)

Step 2. We now apply Crandall-Ishii’s Lemma for each n, with

αn
φn (x, y ) = |x − y |2 .
2
This gives us two sequences of symmetric matrices An and Bn such
that by (1):
−
(αn (xn − yn ), An ) ∈ J¯O
+
u(xn ), (αn (xn − yn ), Bn ) ∈ J¯O v (yn ).

23 of 38
Comparison principle proof (2)

Step 2. We now apply Crandall-Ishii’s Lemma for each n, with

αn
φn (x, y ) = |x − y |2 .
2
This gives us two sequences of symmetric matrices An and Bn such
that by (1):
−
(αn (xn − yn ), An ) ∈ J¯O
+
u(xn ), (αn (xn − yn ), Bn ) ∈ J¯O v (yn ).

Then, by the definition of viscosity subsolution and supersolution

with semijets we get:

F (xn , u(xn ), αn (xn − yn ), An ) ≤ 0 ≤ F (yn , v (yn ), αn (xn − yn ), Bn ).

23 of 38
Comparison principle proof (3)

Step 3. The last result, combined with the assumption of strict

monotonicity, gives us:

γδ ≤ γ(u(xn ) − v (yn ))
≤ F (xn , u(xn ), αn (xn − yn ), An ) − F (xn , v (yn ), αn (xn − yn ), An )
≤ F (yn , v (yn ), αn (xn − yn ), Bn ) − F (xn , v (yn ), αn (xn − yn ), An ).

24 of 38
Comparison principle proof (3)

Step 3. The last result, combined with the assumption of strict

monotonicity, gives us:

γδ ≤ γ(u(xn ) − v (yn ))
≤ F (xn , u(xn ), αn (xn − yn ), An ) − F (xn , v (yn ), αn (xn − yn ), An )
≤ F (yn , v (yn ), αn (xn − yn ), Bn ) − F (xn , v (yn ), αn (xn − yn ), An ).

Lastly we apply the growth assumption and get, by (3):

n→+∞
γδ ≤ π(αn |xn − yn |2 + |xn − yn |) −−−−→ 0,

which leads to a contradiction. It is now only left to prove (3).

24 of 38
Comparison principle proof (4)
Step 4. We now prove (3): O is compact and u − v is upper-
semicontinuous, so there exists x ∗ ∈ O maximizer of u − v and
αn
(u − v )(x ∗ ) ≤ Mαn = u(xn ) − v (yn ) − |xn − yn |2 .
2

25 of 38
Comparison principle proof (4)
Step 4. We now prove (3): O is compact and u − v is upper-
semicontinuous, so there exists x ∗ ∈ O maximizer of u − v and
αn
(u − v )(x ∗ ) ≤ Mαn = u(xn ) − v (yn ) − |xn − yn |2 .
2
So we get, for n → +∞:
αn
ℓ := lim sup |xn − yn |2 ≤ lim sup u(xn ) − v (yn ) − (u − v )(x ∗ )
n→+∞ 2 n→+∞
≤ u(x̂) − v (ŷ ) − (u − v )(x ∗ ).

25 of 38
Comparison principle proof (4)
Step 4. We now prove (3): O is compact and u − v is upper-
semicontinuous, so there exists x ∗ ∈ O maximizer of u − v and
αn
(u − v )(x ∗ ) ≤ Mαn = u(xn ) − v (yn ) − |xn − yn |2 .
2
So we get, for n → +∞:
αn
ℓ := lim sup |xn − yn |2 ≤ lim sup u(xn ) − v (yn ) − (u − v )(x ∗ )
n→+∞ 2 n→+∞
≤ u(x̂) − v (ŷ ) − (u − v )(x ∗ ).

So ℓ < +∞ and lim supn→+∞ |xn − yn | = 0, hence x̂ = ŷ , hence:

αn
0 ≤ lim inf |xn − yn |2 ≤ ℓ ≤ (u − v )(x̂) − (u − v )(x ∗ ) ≤ 0,
n→+∞ 2

which gives us Mαn → (u − v )(x̂).

25 of 38
Viscosity supersolution property (1)
Now we will apply the theory behind viscosity solutions to
stochastic control problems. Let’s recall our Hamiltonian:
1
H(t, x, p, M) = sup[b(x, a) · p + tr(σσ t (x, a)M) + f (t, x, a)]
a∈A 2

Theorem (Viscosity supersolution property)

Suppose the value function v is locally bounded on [0, T ) × Rn ,
that the function f (t, x, a) has quadratic growth, and that it is
continuous in (t, x) for all a ∈ A. Then v is a viscosity
supersolution of the HJB equation:
∂v
− (t, x)−H(t, x, Dx v (t, x), Dx2 v (t, x)) = 0, (t, x) ∈ [0, T )×Rn
∂t

26 of 38
Viscosity supersolution property (2)

A similar result can be obtained in the infinite horizon problem for

the HJB equation:

βv (t, x) − H(t, x, Dx v (t, x), Dx2 v (t, x)) = 0, (t, x) ∈ R+ × Rn

27 of 38
Viscosity supersolution property (2)

A similar result can be obtained in the infinite horizon problem for

the HJB equation:

βv (t, x) − H(t, x, Dx v (t, x), Dx2 v (t, x)) = 0, (t, x) ∈ R+ × Rn

The idea of the theorem’s proof is based on applying Itô’s formula

to the DPP for stopping time t + h getting something like:
Z t+h 
a t,x
v (t, x) + E (−∂t v − L v − f ) (s, Xs , a)ds ≥ 0,
t

with La v = b · Dx v + 21 tr(σσ t Dx2 v ), then by applying the integral

mean and dominated convergence theorems we would prove our
statement. The steps above however require some C 2 regularity for
v which we are lacking, so we need to utilize test functions.

27 of 38
Viscosity supersolution property (3)

Proof. We fix (t, x) ∈ [0, T ) × Rn and a C 2 test function φ such

that:
0 = (v∗ − φ)(t, x) = min (v∗ − φ)(t, x)
(t,x)∈[0,T )×Rn

with v∗ lower-semicontinuous envelope of v .

28 of 38
Viscosity supersolution property (3)

Proof. We fix (t, x) ∈ [0, T ) × Rn and a C 2 test function φ such

that:
0 = (v∗ − φ)(t, x) = min (v∗ − φ)(t, x)
(t,x)∈[0,T )×Rn

with v∗ lower-semicontinuous envelope of v . By definition, there

exists some sequence (tm , xm ) → (t, m) such that:
m→+∞
γm := v∗ (tm , xm ) − φ(tm , xm ) −−−−−→ 0.

28 of 38
Viscosity supersolution property (3)

Proof. We fix (t, x) ∈ [0, T ) × Rn and a C 2 test function φ such

that:
0 = (v∗ − φ)(t, x) = min (v∗ − φ)(t, x)
(t,x)∈[0,T )×Rn

with v∗ lower-semicontinuous envelope of v . By definition, there

exists some sequence (tm , xm ) → (t, m) such that:
m→+∞
γm := v∗ (tm , xm ) − φ(tm , xm ) −−−−−→ 0.

Let α ≡ a ∈ A; we know that α ∈ A(tm , xm ) for all m, and there

exist respective associated processes Xstm ,xm .

28 of 38
Viscosity supersolution property (3)

Proof. We fix (t, x) ∈ [0, T ) × Rn and a C 2 test function φ such

that:
0 = (v∗ − φ)(t, x) = min (v∗ − φ)(t, x)
(t,x)∈[0,T )×Rn

with v∗ lower-semicontinuous envelope of v . By definition, there

exists some sequence (tm , xm ) → (t, m) such that:
m→+∞
γm := v∗ (tm , xm ) − φ(tm , xm ) −−−−−→ 0.

Let α ≡ a ∈ A; we know that α ∈ A(tm , xm ) for all m, and there

exist respective associated processes Xstm ,xm . Lastly we define
τm = inf{s ≥ tm : Xstm ,xm − xm ≥ η}, with η ≥ 0 fixed.

28 of 38
Viscosity supersolution property (4)

Let hm → 0 be a sequence such that hγmm → 0 for m → +∞.

Now we apply the DPP for v (tm , xm ) to θm := τm ∧ (tm + hm ):
Z θm 
v (tm , xm ) ≥ E f (s, Xstm ,xm , a)ds + v (θm , Xθtmm ,xm ) .
tm

29 of 38
Viscosity supersolution property (4)

Let hm → 0 be a sequence such that hγmm → 0 for m → +∞.

Now we apply the DPP for v (tm , xm ) to θm := τm ∧ (tm + hm ):
Z θm 
v (tm , xm ) ≥ E f (s, Xstm ,xm , a)ds + v (θm , Xθtmm ,xm ) .
tm

Since v ≥ v∗ ≥ φ, we get:
Z θm 
tm ,xm tm ,xm
φ(tm , xm ) + γm ≥ E f (s, Xs , a)ds + φ(θm , Xθm ) .
tm

29 of 38
Viscosity supersolution property (4)

Let hm → 0 be a sequence such that hγmm → 0 for m → +∞.

Now we apply the DPP for v (tm , xm ) to θm := τm ∧ (tm + hm ):
Z θm 
v (tm , xm ) ≥ E f (s, Xstm ,xm , a)ds + v (θm , Xθtmm ,xm ) .
tm

Since v ≥ v∗ ≥ φ, we get:
Z θm 
tm ,xm tm ,xm
φ(tm , xm ) + γm ≥ E f (s, Xs , a)ds + φ(θm , Xθm ) .
tm

By Itô’s formula we have:

Z θm Z θm
φ(θm , Xθtmm ,xm ) = φ(tm , xm )+ a
(∂t φ + L φ) ds + σ·∂x φdBs .
tm tm

29 of 38
Viscosity supersolution property (5)

Combining the above results and since σ · ∂x φ is bounded:

 Z θm 
γm 1 a tm ,xm
(1) +E (−∂t φ − L φ − f ) (s, Xs , a)ds ≥ 0.
hm hm tm

30 of 38
Viscosity supersolution property (5)

Combining the above results and since σ · ∂x φ is bounded:

 Z θm 
γm 1 a tm ,xm
(1) +E (−∂t φ − L φ − f ) (s, Xs , a)ds ≥ 0.
hm hm tm

Since xm → x, by a.s. continuity of X we have that ideally:

n o
τm → inf s ≥ t : Xst,x − x ≥ η = t + ω

ω
for some ω > 0, and so for m large enough τm ≥ tm + 2 and we
have θm = tm + hm (since hm → 0).

30 of 38
Viscosity supersolution property (5)

Combining the above results and since σ · ∂x φ is bounded:

 Z θm 
γm 1 a tm ,xm
(1) +E (−∂t φ − L φ − f ) (s, Xs , a)ds ≥ 0.
hm hm tm

Since xm → x, by a.s. continuity of X we have that ideally:

n o
τm → inf s ≥ t : Xst,x − x ≥ η = t + ω

for some ω > 0, and so for m large enough τm ≥ tm + ω2 and we

have θm = tm + hm (since hm → 0).
So, by the mean value theorem and dominated convergence
applied to (1), for m → +∞:

−∂t φ(t, x) − La φ(t, x) − f (t, x, a) ≥ 0 for all a ∈ A.

30 of 38
Viscosity subsolution property (1)

dom(H) := {(t, x, p, M) ∈ [0, T )×Rn ×Rn ×Sn : H(t, x, p, M) < ∞}

Theorem (Viscosity subsolution property)

Suppose the value function v is locally bounded on [0, T ) × Rn
and that there exists G continuous such that:

(t, x, p, M) ∈ dom(H) ⇐⇒ G (t, x, p, M) ≥ 0.

Then v is a viscosity subsolution of the HJB equation:

min{−∂t v (t, x) − H(t, x, Dx v (t, x), Dx2 v (t, x)),

G (t, x, Dx v (t, x), Dx2 v (t, x))} = 0.

31 of 38
Viscosity solution (1)

Theorem (Viscosity solution)

Under the assumption of the previous theorems the value function
v is a viscosity solution of the Hamilton-Jacobi-Bellman variational
inequality:

min{−∂t v (t, x) − H(t, x, Dx v (t, x), Dx2 v (t, x)),

G (t, x, Dx v (t, x), Dx2 v (t, x))} = 0.

Proof. When (t, x) is the minimum of v∗ − φ we have:

−∂t φ(t, x) − H(t, x, Dx φ(t, x), Dx2 φ(t, x)) ≥ 0

so G (t, x, Dx φ(t, x), Dx2 φ(t, x)) ≥ 0.

32 of 38
Viscosity solution (2)
The proof is not complete: we would need to also prove that a
similar result holds for terminal conditions v (T , x) = g (x).

Theorem (Terminal conditions)

Assume f and g to be bounded or have linear growth, and that
there exists G as before, then:
if g is lower-semicontinuous, v∗ (T , ·) is a viscosity
supersolution of

min[v∗ (T , x) − g (x), G (T , x, Dx v∗ (T , x), Dx2 v∗ (T , x))] = 0;

if g is upper-semicontinuous, v ∗ (T , ·) is a viscosity
subsolution of

min[v ∗ (T , x) − g (x), G (T , x, Dx v ∗ (T , x), Dx2 v ∗ (T , x))] = 0.

33 of 38
Comparison principle

Now we would like to find some uniqueness result on viscosity

solutions. For C 2 regularity functions we have a comparison
principle of the form:
Theorem (Comparison principle):
Let U(/V ) ∈ C 1,2 ([0, T ) × Rn ) ∩ C 0 ([0, T ] × Rn ) be a subsolution
(resp. supersolution) with polynomial growth condition to:

−∂t w + βw − H(t, x, Dx w , Dx2 w ) = 0 on [0, T ) × Rn .

If U(T , ·) ≤ V (T , ·) on Rn , then U ≤ V on [0, T ) × Rn

This theorem gives us a good uniqueness solutions for C 2 cases

given initial conditions (through the usual reasoning). What can
we say in viscosity cases?
34 of 38
Strong comparison principle

Theorem (Strong comparison principle):

Let U (/resp. V ) be a u.s.c. viscosity subsolution (resp. l.s.c.
viscosity supersolution) with polynomial growth condition to:

−∂t w + βw − H(t, x, Dx w , Dx2 w ) = 0 on [0, T ) × Rn .

If U(T , ·) ≤ V (T , ·) on Rn , then U ≤ V on [0, T ) × Rn

This grants us the uniqueness result we were looking for.

35 of 38
Strong comparison principle

Theorem (Strong comparison principle):

Let U (/resp. V ) be a u.s.c. viscosity subsolution (resp. l.s.c.
viscosity supersolution) with polynomial growth condition to:

−∂t w + βw − H(t, x, Dx w , Dx2 w ) = 0 on [0, T ) × Rn .

If U(T , ·) ≤ V (T , ·) on Rn , then U ≤ V on [0, T ) × Rn

This grants us the uniqueness result we were looking for.

We can also prove similar comparison principles for the HJB

equations seen above which depend on a function G , however
there is no general way to do so and the approach to the proof
depends on G .

35 of 38
Sketch of the proof (1)

The idea is to use once again Crandall-Ishii’s Lemma, for the same
φ, deducing that:
   
M 0 3 In −In
≤
0 −N ε −In In

and as such for any C , D n × d matrices:

3
tr(CC t M − DD t N) ≤ |C − D|2 .
ε
We argue by contradiction assuming M := sup[0,T ]×Rn (U − V ) > 0.
1
Let φε (t, s, x, y ) = 2ε [|t − s|2 + |x − y |2 ] and let:

Φϵ (t, s, x, y ) = U(t, x) − V (s, y ) − φε (t, s, x, y ).

36 of 38
Sketch of the proof (2)

The proof continues mostly as done before: we choose a bounded

sequence (tε , sε , xε , yε ) maximizing Φϵ for all ε > 0. By using
Crandall-Ishii’s lemma we get M, N as above and such that:
 
1 1
(tε − sε ), (xε − yε ), M ∈ J¯(0,T+
) U(tε , xε )
ε ε
 
1 1 −
(tε − sε ), (xε − yε ), N ∈ J¯(0,T ) V (sε , yε )
ε ε

37 of 38
Sketch of the proof (3)

Then we use the characterization of U, V as viscosity solutions

through superjets, the Lipschitz conditions on b, σ and the uniform
continuity of f to get:
1
β[U(tε , xε ) − V (sε , yε )] ≤H(tε , xε , (xε − yε ), M)+
ε
1 2
−H(sε , yε , (xε − yε ), N) ≤µ(|tε − sε | + |xε − yε | + |xε − yε |2 ),
ε ε
and from ε → 0 we get βM ≤ 0, a contradiction.

38 of 38