## Section: New Results

### Stochastic processes, queueing, control theory and game theory

Participants : Eitan Altman, Konstantin Avrachenkov, Amar Azad, Alain Jean-Marie.

#### Convergence of rolling horizon control

In collaboration with E. Della Vecchia and S. Di Marco (National University of Rosario, Argentina), A. Jean-Marie has investigated the performance of the Rolling Horizon heuristic for optimal stochastic control when the optimization criterion is the long-term average expected gain [46] . They have shown that convergence occurs under quite general assumptions, weaker than the usual “unichain” assumption. As a side result, they have shown that a stopping rule for the Value Iteration algorithm, conjectured by Puterman, is not correct in general.

#### Advances in queueing theory

In [27] , K. Avrachenkov in collaboration with U. Yechiali (Tel Aviv Univ.), considers systems of tandem blocking queues having a common retrial queue. The model can represent dynamics of short TCP transfers in the Internet. Analytical results are available only for a specific example with two queues in tandem. They propose approximation procedures involving simple analytic expressions, based on mean value analysis (MVA) and on fixed point approach (FPA). The mean sojourn time of a job in the system and the mean number of visits to the orbit queue are estimated by the MVA which needs as an input the fractions of blocked jobs in the primary queues. The fractions of blocked jobs are estimated by FPA.

#### Analysis of DPS in overload and applications to TCP

In [21] , E. Altman, T. Jimenez (LIA/Univ. Avignon) and D. Kofman (Télécom ParisTech) study the rate of growth of the delays as well as the rate of growth of the population size of sessions in a network at overload, which they model as a DPS (Discriminatory Processor Sharing). They obtain a fixed point equation that allows them to compute the growth rate for any stationary ergodic service and arrival time process. They then study how suitable are the results for describing the session level performance of file transfers in the Internet.

#### Markov decision evolutionary games

Evolutionary games concern the evolution of populations that interact with each other through many simultaneous pairwise interactions. The result of each such local interaction is determined by the actions of the individuals involved. In [20] , E. Altman and Y. Hayel (LIA/Univ. Avignon) extend the theory of evolutionary games to include also a notion of a state of each individual; the results (and payoff) of the interactions are now determined not only by the actions taken by individuals but also by their individual states. The actions and the current individual states further determine the transition probabilities of these states. The theory is applied to power control in wireless networks. This application as well as applications to contributions to the theory evolutionary games are surveyed in [29] by H. Tembine (Supelec), E. Altman, R. El-Azouzi and Y. Hayel (LIA/Univ. Avignon).

#### Singular perturbation theory

In [26] , K. Avrachenkov, in collaboration with V. Ejov and J. Filar (Univ. South Australia), studies multivariate perturbations of algebraic equations. In general, it is not possible to represent the perturbed solution as a Puiseux-type power series in a connected neighborhood. For the case of two perturbation parameters, the authors provide a sufficient condition that guarantees such a representation. Then, the authors extend this result to the case of more than two perturbation parameters. The study is motivated by the perturbation analysis of a weighted random walk on the Web Graph. As an instance of the latter, the stationary distribution of the weighted random walk, the so-called Weighted PageRank, may depend on two (or more) perturbations.

#### Game theory, altruism and the degree of cooperation

In [40] , A. Azad and E. Altman, in cooperation with R. El-Azouzi (LIA/Univ. Avignon), introduce a parametrized level of cooperation. The utility of a player is assumed to be a weighted average of performance measures of other players. Varying the weight gives all the cooperation spectrum from non-cooperation till altruism. The authors apply the concept to routing games and investigate the properties of the equilibria as a function of the degree of cooperation.