Project-Sydoco:Numerical methods for HJB equation

Inria / Raweb 2006
Presentation of the Project Sydoco

Section: New Results

Numerical methods for HJB equation

Splitting

Participants : F. Bonnans, H. Zidani, E. Ottenwaelter.

We have continued the study of splitting algorithms for solving the HJB equation of stochastic control. We have clarified the issue of monotonicity and consistency of such algorithms. It appears unfortunately that monotonicity and consistency occur only under quite restrictive hypothesis.

A numerical method for a stochastic impulse control problem. Some results on Howard algorithm

Participants : F. Bonnans, S. Maroso, H. Zidani.

In the framework of the thesis of S. Maroso, we have studied an implementable scheme for solving the HJB equation for stochastic impulse control problem. Our scheme is based on the cascade approach that we have already used in [17] , to study the error estimates for the numerical approximation of the impulse HJB equation. At each (each) step of our algorithm, we have to solve an obstacle problem. We suggest to perform this resolution by doing a given number of iterations of policy algorithm (also called Howard algorithm).

We also study some convergence results for the policy algorithm. More precisely, we give a simple proof of a superlinear convergence of the policy iterations when applied to solve a problem in the following setting:

$Im1 ${{Max}_{\#945 \#8712 \#119964 ^N}(A(\#945 )X-f{(\#945 )};}$$

here, the output is the vector X $\in$ IR^N , $Im2 $\#119964 $$ being a compact set, and for $Im3 ${\#945 \#8712 \#119964 ^N}$$ , A( $\alpha$ ) is a monotone matrix and f( $\alpha$ ) is a vector in IR^N . The main idea in our proof is the formulation of the Howard algorithm as a semi-smooth Newton's method applied to find the zero of the function F defined by:

$Im4 ${{F(X):}={Max}_{\#945 \#8712 \#119964 ^N}(A(\#945 )X-f{(\#945 )}.}$$

We prove also that the function F is differentiable in a weak sense (slant differentiability) [38] , [36]

On the other hand, we prove that the Howard algorithm used for solving an obstacle problem:

M a x (MX-b, X- $\upper_psi$ ) = 0,

is strictly equivalent to the Primal-dual algorithm introduced by Ito-Kunich [36] . For more details, see [10] .

Numerical approximation for a super-replication problem

Participants : S. Maroso, H. Zidani.

In a financial market, consisting in a non-risky asset and some risky assets, people are interested to study the minimal initial capital needed in order to super-replicate a given contingent claim, under gamma constraints. Many authors have studied this problem in theoretical point of view [35] , [33] , [34] .

In collaboration with O. Bokanowski(Lab. Jacque-Louis Lions, Paris 7), and B. Bruder(Lab. Probabilités et modèles alátoires, Paris 7. Also at Soc. Générale), we study a super-replication problem in dimension 2. The main difficulty for this problem comes from the non-boundedness of the control set. First we give a characterisation of the value function $Im5 $\#977 $$ as unique viscosity solution of an HJB equation:

$Im6 ${\#923 ^-\mfenced o=( c=) {J(t,x,y,D\#977 (t,x,y),}D^2{\#977 (t,x,y))}={0,~x,y\#8712 (0,+\#8734 ).}}$$

(1)

where J is a symetric matrix differential operator associated to the Hamiltonian, and where $\upper_lambda$ ^-(J) denotes the smallest eigenvalue of J.

The advantage of the above HJB equation lies on the fact that the operator J does not depend on the control variable, but the "non standard" form of the equation could lead us to think that it is not-useful. However, from standard calculations, we obtain a simple formulation of (1 ) in the following form:

$Im7 ${min_{\#8214 \#945 \#8214 =1}\#945 ^T{J(t,x,y,D\#977 (t,x,y),}D^2\#977 (t,x,y))\#945 =0,~x,y\#8712 {(0,+\#8734 )}.}$$

(2)

In this new formulation the variable $\alpha$ can be seen as a bounded control variable.

We study an approximation scheme for the equation (2 ) based on the generalized finite differences algorithm introduced in [6] , [4] . We prove the existence, uniqueness of a bounded discrete solution. We also verify the monotonicity and stability of the scheme. Moreover, we give a consistance error approximation. Then by using the same arguments as in [30] , we prove the convergence of the discrete solutions towards the value function $Im5 $\#977 $$ , when the discretization step size tends to 0.

A preliminary version of this work is presented in the thesis of S. Maroso, while a complete version will be submitted as an INRIA report.

Logo Inria