Section: New Results

Exit time for PDMP's

Participants : Adrien Brandejsky, Benoîte de Saporta, François Dufour.

The aim of this work is to propose a practical numerical method to approximate the survival function and the moments of the exit time for a piecewise-deterministic Markov process thanks to the quantization of a discrete-time Markov chain naturally embedded within the continuous-time process.

Numerical computation of the moments of the exit time for a Markov process has been studied by K. Helmes, S. Röhl and R.H. Stockbridge in  . Starting from an assumption related to the generator of the process, they derive a system of linear equations satisfied by the moments. In addition to these equations, they include finitely many Hausdorff moment conditions that are also linear constraints. This optimization problem is a standard linear programming problem for which many efficient softwares are available. J.-B. Lasserre and T. Prieto-Rumeau introduced in  a similar method but they improved the efficiency of the algorithm by replacing the Hausdorff moment conditions with semidefinite positivity constraints of some moment matrices. Nevertheless, their approach cannot be applied to PDMP's because the assumption related to the generator of the process is generally not satisfied. In  , M.H.A. Davis gives an iterative method to compute the mean exit time for a PDMP but his approach involves solving a large set of ODEs whose forms are very problem specific, depending on the behavior of the process at the boundary of the state space. Besides, in the context of applications to reliability, it seems important to study also the distribution of the exit time.

In our approach, the first step consists in expressing the j -th moment (respectively the survival function) as the last term of some sequence (respectively ) satisfying a recursion equation pk + 1, j = (pk, j) (respectively pk + 1 = (pk) ). The natural way to deal with these problems is to follow the idea developed in  namely to rewrite the recursions in terms of an underlying discrete-time Markov chain and to replace it by its quantized approximation. The definitions of the recursive sequences (pk, j)k and (pk)k involve some discontinuities related to indicator functions but as in  , we show that they actually happen with small enough probability. However, an important feature that distinguishes the present work from  and which prevents a straightforward application of the ideas developed therein, is that the mapping defining the recursions pk + 1, j = (pk, j) and pk + 1 = (pk) is not Lipschitz continuous. One of the main results of this paper is to overcome this difficulty by using sharp feature of the quantization algorithm. We are able to prove the convergence of the approximation scheme. Moreover, in the case of the moments, we even obtain bounds for the rate of convergence. It is important to stress that these assumptions are quite reasonable judging from the situations met in applications.

In addition, our method is highly flexible. Indeed, as pointed out in  , a quantization based method is obstacle free in the sense that it provides, once and for all, a discretization of the process independently of the set we want to exit from. Consequently, the approximation schemes for both the moments and the distribution of the exit time are flexible w.r.t. to this exit set. This work has been submitted for publication (see http://hal.inria.fr/hal-00546339 ).

Logo Inria