Section: New Results

Numerical method for impulse control of PDMP's

Participants : Benoîte de Saporta, François Dufour, Huilong Zhang.

We have also extended our results on optimal stopping to the impulse control problem. An impulse control strategy consists in a sequence of single interventions introducing a jump of the process at some controller-specified stopping time and moving the process at that time to some new point in the state space. Our impulse control problem consists in choosing a strategy (if it exists) that minimizes the expected sum of discounted running and intervention costs up to infinity, and computing the optimal cost thus achieved. Many applied problems fall into this class, such as inventory problems in which a sequence of restocking decisions is made, or optimal maintenance of complex systems with components subject to failure and repair.

Impulse control problems of PDMP's in the context of an expected discounted cost have been considered in [46] , [48] , [52] , [53] , [57] . Roughly speaking, in [46] the authors study this impulse control problem by using the value improvement approach while in [48] , [52] , [53] , [57] the authors choose to analyze it by using the variational inequality approach. In [46] , the authors also consider a numerical procedure based on the iteration of the single-jump-or-intervention operator and a uniform discretization of the state space. Our approach is also based on the iteration of the single-jump-or-intervention operator, but we want to derive a convergence rate for our approximation. Our method does not rely on a blind discretization of the state space, but on a discretization that depends on time and takes into account the random nature of the process. Our approach involves a quantization procedure. Roughly speaking, quantization is a technique that approximates a continuous state space random variable X by a a random variable taking only finitely many values and such that the difference between X and is minimal for the Lp norm.

Although the value function of the impulse control problem can be computed by iterating implicit optimal stopping problems, see [46] , [47] , from a numerical point of view the impulse control is much more difficult to handle than the optimal stopping problem. Indeed, for the optimal stopping problem, the value function is computed as the limit of a sequence (vn) constructed by iterating an operator L . This iteration procedure yields an iterative construction of a sequence of random variables vn(Zn) (where (Zn) is an embedded discrete-time process). This was the keystone of our approximation procedure. As regards impulse control, the iterative construction for the corresponding random variables does not hold anymore. This is mostly due to the fact that not only does the controller choose times to stop the process, but they also choose a new starting point for the process to restart from after each intervention. This makes the single-jump-or-intervention operator significantly more complicated to iterate that the single-jump-or-stop operator used for optimal stopping. We managed to overcome this extra difficulty by using two series of quantization grids instead of just the one we used for optimal stopping. This work has been submitted for publication (see http://hal.inria.fr/hal-00541413 ).

Logo Inria