## Section: Research Program

### Optimal control and zero-sum games

The dynamic programming approach allows one to analyze one or two-player dynamic decision problems by means of operators, or partial differential equations (Hamilton–Jacobi or Isaacs PDEs), describing the time evolution of the value function, i.e., of the optimal reward of one player, thought of as a function of the initial state and of the horizon. We work especially with problems having long or infinite horizon, modelled by stopping problems, or ergodic problems in which one optimizes a mean payoff per time unit. The determination of optimal strategies reduces to solving nonlinear fixed point equations, which are obtained either directly from discrete models, or after a discretization of a PDE.

**The geometry of solutions of optimal control and game problems**
Basic questions include, especially for stationary or ergodic problems, the understanding of existence and uniqueness conditions for the solutions of dynamic
programming equations, for instance in terms of controllability
or ergodicity properties, and more generally the understanding of the structure
of the full set of solutions of stationary Hamilton–Jacobi PDEs and of the set of optimal strategies. These issues are already challenging
in the one-player deterministic case, which is an application
of choice of tropical methods, since the Lax-Oleinik semigroup, i.e.,
the evolution semigroup of the Hamilton-Jacobi PDE, is a linear
operator in the tropical sense.
Recent progress in the deterministic case
has been made by combining dynamical systems and PDE techniques (weak KAM theory [72]),
and also using metric geometry ideas (abstract boundaries can be used to represent the sets of solutions [86], [4]).
The two player case is challenging, owing to the lack of compactness of the analogue of the Lax-Oleinik semigroup and to a richer geometry. The conditions of solvability of ergodic problems for games (for instance, solvability of ergodic Isaacs PDEs), and the representation of solutions are only understood in special cases, for instance in the finite state space case, through tropical geometry and non-linear Perron-Frobenius methods [38], [41], [3].

**Algorithmic aspects: from combinatorial algorithms to the attenuation of the curse of dimensionality**
Our general goal is to push the limits of solvable models by means
of fast algorithms adapted to large scale instances.
Such instances arise from discrete problems,
in which the state space may so large that
it is only accessible through local oracles (for instance,
in some web ranking applications, the number of states may
be the number of web pages) [73]. They also arise
from the discretization of PDEs, in which the number
of states grows exponentially with the number of degrees
of freedom, according to the “curse of dimensionality”.
A first line of research is the development of *new
approximation methods for the value function*. So far, classical approximations
by linear combinations have been used, as well as approximation
by suprema of linear or quadratic forms, which have
been introduced in the setting of dual dynamic programming
and of the so called “max-plus basis methods” [74]. We believe that more concise
or more accurate approximations may be obtained by unifying
these methods.
Also, some max-plus basis methods have
been shown to *attenuate the curse of dimensionality*
for very special problems (for instance involving switching) [97], [78].
This suggests that the complexity of control or games problems may be measured
by more subtle quantities that the mere number of states, for instance,
by some forms of metric entropy (for example, certain large scale
problems have a low complexity owing to the presence of decomposition
properties, “highway hierarchies”, etc.).
A second line of of our research is the development
of *combinatorial algorithms*,
to solve large scale zero-sum two-player problems
with discrete state space. This is related to current
open problems in algorithmic game theory. In particular,
the existence of polynomial-time algorithms for games
with ergodic payment is an open question.
See e.g. [43] for a polynomial time average complexity result derived
by tropical methods.
The two lines
of research are related, as the understanding of the geometry of solutions
allows to develop better approximation or combinatorial algorithms.