TAO (
*Thème Apprentissage et Optimisation*) is a joint project inside PCRI, including researchers from INRIA and the LRI team
*I & A – Inférence et Apprentissage*(CNRS and University of Paris Sud), located in Orsay.

Data Mining (DM) has been identified as one of the ten main challenges of the 21st century (MIT Technological Review, fev. 2001). The goal is to exploit the massive amounts of data produced in scientific labs, industrial plants, banks, hospitals or supermarkets, in order to extract valid, new and useful regularities. In other words, DM resumes the Machine Learning (ML) goal, finding (partial) models for the complex system underlying the data.

DM and ML problems can be set as optimization problems, thus leading to two possible approaches. Note that this alternative has been characterized by H. Simon (1982) as follows.
*In complex real-world situations, optimization becomes approximate optimization since the description of the real-world is radically simplified until reduced to a degree of complication that
the decision maker can handle. Satisficing seeks simplification in a somewhat different direction, retaining more of the detail of the real-world situation, but settling for a satisfactory,
rather than approximate-best, decision.*

The first approach is to simplify the learning problem to make it tractable by standard statistical or optimization methods. The alternative approach is to preserve as much as possible the genuine complexity of the goals (yielding “interesting” models, accounting for prior knowledge): more flexible optimization approaches are therefore required, such as those offered by Evolutionary Computation.

Symmetrically, optimization techniques are increasingly used in all scientific and technological fields, from optimum design to risk assessment. Evolutionary Computation (EC) techniques, mimicking the Darwinian paradigm of natural evolution, are stochastic population-based dynamical systems that are now widely known for their robustness and flexibility, handling complex search spaces (e.g. mixed, structured, constrained representations) and non-standard optimization goals (e.g. multi-modal, multi-objective, context-sensitive), beyond the reach of standard optimization methods.

The price to pay for such properties of robustness and flexibility is twofold. On one hand, EC is tuned, mostly by trials and errors, using quite a few parameters. On the other hand, EC generates massive amounts of intermediate solutions. It is suggested that the principled exploitation of preliminary runs and intermediate solutions, through Machine Learning and Data Mining techniques, can offer sound ways of adjusting the parameters and finding short cuts in the trajectories in the search space of the dynamical system.

The overall goals of the project are to model, to predict, to understand, and to control physical or artificial systems. The central claim is that Learning and Optimisation approaches must be used, adapted and integrated in a seamless framework, in order to bridge the gap between the system under study on the one hand, and the expert's goal as to the ideal state/functionality of the system.

Specifically, our research context involves the following assumptions:

The systems under study range from large-scale engineering systems to physical or chemical phenomenons, including games. Such systems, sometimes referred to as
*complex systems*, can hardly be modelled based on first principles due to their size, their heterogeneity and the incomplete information aspects involved in their behaviour.

Such systems can be observed; indeed selecting the relevant observations and providing a reasonably appropriate description thereof is part of the problem to be solved. A further assumption is that these observations are sufficient to build a reasonably accurate model of the system under study.

The available expertise is sufficient to assess the system state, and any modification thereof, with respect to the desired states/functionalities. The assessment function is usually not a well-behaved function (differentiable, convex, defined on a continuous domain, etc), barring the use of standard optimisation approaches and making Evolutionary Computation a better suited alternative.

In this context, the objectives of TAO are threefold:

using Evolutionary Computation (EC) and more generally Stochastic Optimisation to support Machine Learning (ML);

using Statistical Machine Learning to support Evolutionary Computation;

investigating integrated ML/EC approaches on diversified and real-world applications.

Due to the unavoidable shift of the scientific environement and people interest after 4 years of activity, the detailed implementation of those objectives have been slightly revised since the initial project proposal 4 years ago, and updated lines of research will be described in next section .

The MoGo program (see sections and ) is one of the most prominent successes of TAO in the period; MoGo repeatedly won international computer-go contests in 2006 and 2007; it was the first computer-go program to ever reach 2,000 ELO, and the first one to win over a (3e dan) human player (on 9x9 goban); it still is ranked best of the computer go programs. Its port onto a heavily parallel platform (collaboration with Bull), should increase this domination and open new horizons in computer-go.

This section is the prospective part of the report written for the Evaluation Seminar that took place in November 2007 for the INRIA theme COG A, to which TAO belongs. Because this was a unique opportunity to re-think the objectives of the project after 4 years of activity, it naturally becomes the Scientific Foundations of TAO activities for the next 4 years.

Four main objectives have been identified, related to scientific bottlenecks of Machine Learning and Stochastic Optimisation. The first one, illustrated by the results
discussed in sections
and
, is devoted to the search of representations with desirable properties. The second one, exploiting the prominent successes
obtained with Multi-Armed Bandit algorithms and MoGo (section
), considers the various challenges raised by extending MAB algorithms to dynamic contexts and hostile environments. The third
one will investigate the conceptual and algorithmic shift required to deal with modern computing architectures. The fourth one is concerned with
*Crossing the Chasm*
*Crossing the Chasm: Marketing and Selling High-Tech Products to Mainstream Customer*, 1991

These objectives are relevant to the main research projects in our agenda. For the sake of completeness the projects starting end 2007 or in 2008 are listed below as they do not appear in section :

*Adaptive Combinatorial Search*, joint project with Microsoft Research (M. Schoenauer and Y. Hamadi, Microsoft Research Cambridge, coordinators), aims at the automatic tuning of search
and optimization algorithms, from heuristics to meta-heuristics (more precisely Evolutionary Computation). Both off-line and on-line tunings will be investigated.

SYMBRION, an FP7 IP (Integrated Project), coordinated by Sergey Kornienko (Univ. Stuttgart), involving 10 partners from Robotics (Electronics and Mechanics), Evolutionary Biology, and Computer Science (working on bio-inspired complex systems). Integrating hardware and software design, Symbrion IP aims at designing autonomous swarm robots. The software will involve both time-scales of evolutionary learning and on-line learning, in direct connection with TAO research themes.

Observatory of the EGEE Grid (WP in EGEE-III, C. Germain coordinator). Resuming earlier studies in EGEE-II, the Grid Observatory will integrate the collection and publication of traces of the EGEE grid and users with the development of models and ontology of the domain knowledge.

DigiBrain, a Digiteo project, coordinated by J.-D. Muller (CEA-LIST) and M. Sebag, involves CEA, INRIA, LRI and Neurospin (S. Dehaene and J.-B. Poline). DigiBrain will investigate EEG-based interfaces for neuroscience studies and Human Computer Interaction.

The last years have seen some breakthroughs related to the ML search space, including Deep Networks (DN

Interestingly, the distinction between the search space (referred to as genotypic space) and the solution space (referred to as phenotypic space) has long been identified as a main source of
effectiveness for Evolutionary Computation
*The Genetic Basis of Evolutionary Change*, Columbia University Press, 1974.

Taking advantage of the statistical learning and evolutionary cultures in TAO, our first research objective will be to analyze and study the diverse frameworks enabling a
**compact description of complex solutions through procedural heuristics**, referred to as Deep Representations (DRs). The theoretical study will focus on the following two aspects:

Expressivity/tractability tradeoff.

DRs allow a huge solution space to be searched through exploring a comparatively restricted and well identified search space, which either offers some performance
guarantees, or was found the only feasible way to obtain any result at all
*Genetic Programming III: Automatic Synthesis of Analog Circuits*. MIT Press, 1999

Stability/versatility tradeoff.

The feasibility of learning/optimization requires some stability of the search landscape, meant as most genotypic changes result in little phenotypic differences. In the
meanwhile, the search space should offer “sufficiently” many shortcuts toward various regions of the solution space. This property, referred to as versatility, implies that additional
information enables efficient jumps in the phenotypic space, making the most of efficient active learning or exploration strategies. The stability/versatility tradeoff will be studied in
the spirit of active learning

In an application perspective, the search for deep representations is relevant to the on-going Gennetec project (investigating Gene Regulatory Networks in a Genetic Programming perspective), and to Symbrion IP (as the target representation should allow learning at different time scales, e.g. involving both evolutionary optimization and on-line learning).

The blossoming use of Multi-Armed Bandit (MAB) algorithms to revisit reinforcement learning

Several extensions of MAB algorithms and analysis have been identified as theoretical and applicative priorities on our research agenda:

A first extension is required to deal with
**dynamic environments**, relaxing the assumption of iid rewards for each option (bandit arm). Let us consider for instance the EvE tradeoff at the core of evolutionary computation, of
game strategies
, of news recommendation
*Online Trading of Exploration and Exploitation Workshop, NIPS*, 2006.

A second extension regards
**multi-variate bandits**. In quite a few application domains, some side information is available (e.g. the user profile in a news recommendation context) and can be used to handle the
EvE tradeoff more efficiently. In the MoGo system, the so-called RAVE (
*Rapid Action Value Estimate*) heuristics provides additional estimates of the move values; significant improvement of MoGo has been obtained by exploiting this additional, most often
strongly biased, side information. Notably, multi-variate bandit algorithms have been acknowledged a prioritary research direction in the PASCAL-2 roadmap

Thirdly, the extension of MAB algorithms to the
**bounded rationality**framework, e.g., increasing the number of options and considering a short-term time horizon, is a both theoretical and applicative challenge. Quite a few
application domains involve many options (e.g. circa 400 arms in computer-go, and infinitely many in continuous frameworks); further more, in games or planning, the stress is put on the
short term performances (as opposed to, the asymptotic ones). We have developed efficient anytime algorithms, extending Berry et al.
*Ann. Stat.*
**25**(5):2103-2116, 1997.

A fourth research perspective related to MAB is concerned with
**multi-objective settings**. For instance in autonomous robot control, every option can be assessed along several criteria, such as its value (to which extent the option is instrumental
to reach the robot goal) and its risks (to which extent the option can dammage the robot integrity). Although multi-objective optimization can always be cast as a mono-objective
optimization problem (e.g. classically considering the weighted sum of the objective functions as single objective), it is believed that multi-objective bandits correspond to a relevant and
daring extension of MAB algorithms. On the one hand, this extension aims at finding optimal, e.g. controlled risk-taking, decision strategies; on the other hand it requires to extend the
regret definition (e.g. cumulative distance to the Pareto front).

These research themes are directly relevant to
*Microsoft-TAO project*, SYMBRION IP and
*Autonomic Computing*.

Typically, the online learning of hyperparameters tackled by
*Microsoft-TAO project*can be formalized as a MAB problem; bounded rationality is similarly relevant at least in the calibration stage of the current application. The multi-objective
aspect is also relevant as the algorithmic performance can usually be assessed along quite a few independent criteria (e.g. time-to-solution, memory usage, solution accuracy).

Independently, optimal decision making under bounded resource constraints is relevant to SYMBRION IP. Likewise, the mid-term goal of Autonomic Computing is to deliver satisficing job
schedulers. In both cases, as experiments are done
*in situ*the learning algorithm must find a way to preserve the system integrity, and limit the risks incurred by any system unit.

Notable breakthroughs have revolutionized computer science in the last decade, along several perspectives: hardware, communication/networks, and usage. With the evolution of means and ends
(multi-core/multi-thread machines; grid systems; distributed/ubiquitous computing; resource-aware/anytime/scalable algorithms; ambient/pervasive intelligence), the software world gradually
becomes more aware that new algorithmic paradigms are required to make the most of new architectures, to handle new demands, and to sail towards New Intelligence Frontiers

While TAO is already interested in modern computing as a source of applications (e.g. through
*Autonomic Computing*and our collaboration with Alchemy
), this third direction of research aims at developing a different approach of algorithms, along the
*Computational Thinking*

A first theme in this research direction aims at the global optimization of the information processing chain

Secondly, Computational Thinking advocates the ability of dealing with large complex (algorithmic) systems without understanding every detail thereof, which shall be referred to as Smart Black Box Design. This second theme of research is relevant to the search of Deep Representations and the Gennetec project, already mentioned in section . It also includes the studies related to complex systems (section ), where tools from statistical physics are used to provide a manageable model at the macro-scale of a system described at the micro-scale. Last, but not least, the DigiBrain project aims at creating the means for an effective interaction with the user without spelling the interaction rules.

Countless studies have underlined the fact that many a good scientific and algorithmic advances fail to make it outside research labs. Such failures are often blamed on the Knowledge Barrier, the entry ticket people have to pay in order to master new techniques because of their many options and parameters, because of their flexibility and versatility. Parameter adjustment is at the core of most if not all TAO applicative studies in Numerical Engineering .

Considerable efforts are deployed to anticipate this usage barrier when designing algorithms or devices. One approach aims at building “plug-and-play” variants, allowing naive users to
benefit from 90% of the algorithm potentialities. It is generally deemed however that some form of adaptation to the problem at hand is required to deliver robust results; in most application
domains, the search for “the” killer algorithm has been acknowledged to be hopeless. Alternatively, one might want to build a meta-layer topping a set of algorithms, selecting the best
algorithm to use depending on the problem at hand

The fourth research direction of TAO will investigate two approaches for Crossing the Chasm, respectively based on online and offline hyperparameter tuning/learning.

The online tuning approach includes both self-adaptation heuristics (dating back to the 90's
*Parameter Setting in Evolutionary Algorithms*, Springer Verlag 2007.*Theoretical Computer Science*, 334(1-3):35–69, 2005.

The second approach, concerned with offline learning, resumes the Phase-Transition studies pioneered by TAO in collaboration with L. Saitta and A. Giordana (Univ. del
Piemonte Orientale), (see section
). Based on the appropriate order parameters, the phase transition framework allows for building the “competence map”
describing the average algorithm behaviour. Clearly, such competence maps make it easy to achieve meta-learning
**J. Maloberti and M. Sebag**. Fast theta-subsumption with constraint satisfaction algorithms.
*Machine Learning Journal*, 55:137–174, 2004.

The fundamental bottleneck, i.e. designing relevant order parameters, has been tackled empirically so far. Further studies will use the statistical physics tools
*Spin glass theory and beyond*, World Scientific, 1987.*Generalized belief propagation*, Advances in Neural Information Processing Systems (NIPS 2001).*Phys. Rev. Letters E*
**66**

This direction of research subsumes several on-going projects, specifically Evotest and
*Microsoft-TAO project*where the goal is to produce off-the-shelf algorithms and to enforce technology transfer. In contrast, many previous
or current projects (e.g. GENNETEC) have enforced technology transfer ... manually.

The mainstream applications of TAO are, since its creation, autonomous robot control, medical applications, and engineering applications. Software robotics is still an active application domain (see section , and several applications involving numerical engineering are on-going (section ).

However, most medical applications have been hindered as the dialogue with experts could not take place to the required extent. In the neighbour domain of neurosciences (in the context of
Neurodyne ACI), the collaboration was effective
*20
^{th}Int. Joint Conf. on Artificial Intelligence (IJCAI-07)*, pages 859-864, 2007.

Moreover, two new fields of applications have appeared, due to the arrival of new staff on the project:

Autonomic Computing applications have started to become an important part of TAO applications, since Cecile Germain-Renaud, now Professor at Université Paris-Sud, joined TAO in 2005, and Balázs Kégl was hired at LAL in Orsay and became a full associate member of the team (see section ).

the study of Social Systems, more precisely economical multi-agent models and road traffic models, have started at TAO, following the arrival of respectively Philippe Caillou and Cyril Furtlehner.

All those applications are described in detail along the text, most of them in section .

MoGo (55000 lines of code in C++) is currently one of the best Computer-Go programs worldwide (including advanced options as multithreading and now message-passing parallel version) (see
Section
. Only the binary code is released, with hundreds of downloads

**Abstract**: GUIDE is a graphical user interface for easy Evolutionary Algorithm design and coding. It allows the user to describe its genome (the structure that will evolve) graphically,
represented as a tree, using containers and elementary types (booleans, integers, real numbers and permutations). All representation-dependent operators (initialization, crossover and mutation)
can then be defined either using default values, built bottom-up from the elementary types, or user-defined operators. Developing a prototype for a new search space involving complex structures
has now become a matter of minutes.

GUIDE was programmed in JAVA by James Manley during the 6 months of his DESS stage in 2004. It is a follow-up of a previous tool developed in collaboration with Pierre Collet in the DREAM
project (
http://

GUIDE has been chosen as the evolutionary basis for the EvoTest project: testing a given program means feeding it with data of a specific structure. In this context, the goal of EvoTest is to automatically evolve test data, relying on an automatic code generator that only requires a description of the structure of the data to evolve – and this is precisely what GUIDE is doing.

The main changes in GUIDE in 2007 have been a complete cleaning of the code (preserving, but not increasing, the functionalities). Consequently, it is now much easier to generate code for different libraries, and the next version will handle EO as well as ECJ (Evolutionary COmputation in Java). Moreover, because of the arrival of Luis DaCosta as developper, GUIDE is now available on GForge as an Open Source software . Also, to fit in the complete loop of Automated testing, GUIDE has been interfaced with the partners' systems through a TPTP interface.

**Abstract**: Simbad is an open source Java 3D robot simulator for scientific and educational purposes (Authors: Louis Hugues and Nicolas Bredèche). Simbad embeds two stand-alone additional
packages: (1) a Neural Network library (PicoNode) and (2) an Artificial Evolution Engine (PicoEvo). The Simbad package is targeted towards Autonomous Robotics and Evolutionary Robotics for
research and education. The packages may be combined or used alone. In the scope of Research in Evolutionary Robotics, the Simbad package package helps quick development of new approaches and
algorithms thanks to the complete and easy-to-extend libraries. Real-world interface can be easily written to transfer simbad controllers to real robots (the Khepera interface is available).
The open source nature of the project combined with easy-to-understand code makes it also a good choice for teaching Autonomous and Evolutionary Robotics. Simbad is used in several AI and
robotics courses: IFIPS engineering school (4th and 5th year) ; Master 1 at Université Paris-Sud ; Modex at Ecole Polytechnique.

Please refer to :
http://

**Abstract**:

PuppetMaster is an open source C++ 3d robotic simulation framework for scientific and educational purposes. It allows to describe simulation scenarios, robot morphologies and behaviors as a C++ plugin. A visualizer makes it possible to see a plugin in action. The simulation is based on realistic physical simulations, so the range of the representable robots and simulations scenarii covers all the practical cases. It allows rapid prototyping of both control algorithm and robot morphology. Combined with a numerical optimization framework, it allows fully automatic design of robots, with simulations scenarios as fitness measure. PuppetMaster was used to design a robot control algorithm independent from the morphology, allowing tests on snake-like and multi-legged robots.

**Abstract**: Django is an algorithm of theta-subsumption of Datalog clauses, written in C by Jerome Maloberti and freely available under the GNU Public License. This algorithm is an exact
one, with a gain of two or three orders of magnitude in computational cost over other theta-subsumption algorithms. Django uses Constraint Satisfaction techniques such as Arc-Consistency,
Forward-Checking and M.A.C. (Maintaining Arc-Consistency) and heuristics based on the First Fail Principle.

Django has been widely used and cited in the literature (coll. with the Yokohama University, Japan, U. of Tufts in Arizona, USA, U. of Bari, Italy).

http://

**Abstract**: OpenDP is an open source code for stochastic dynamic programming
*NIPS Workshop on Machine Learning Open Source Software*, 2006.

The merit of the OpenDP platform is twofold. On one hand, while many of the above algorithms are well-known, their use in a dynamic programming framework is new. On the other hand, such a
systematic comparison of these algorithms on general benchmarks did not exist in the literature of stochastic dynamic programming, where many papers only consider one learning method, not
necessarily in the same conditions than other published results. These thorough experimentations inspired some theoretical work in progress about the criteria for learning in dynamic
environments, noting that cross-validation is neither satisfactory (for example the
^{2}parameter in Gaussian SVM chosen by cross-validation is usually too small in the context of dynamic programming) nor fast enough in that framework.

See main page at
http://

At the core of Machine Learning is the representation of the problem domain. Building an appropriate representation, aka Feature Extraction

Some results previous to TAO regarding Feature Selection exploits the stochasticity of EC-based learning: The ROGER algorithm (
*ROC-based GEnetic learneR*)

Another kind of ensemble-based feature selection has been recently devised and applied for DNA microarrays analysis
, focusing on the notion of Type I and Type II errors (distinguishing relevant and irrelevant features using feature
rankings based on independent criteria)
*Multiple Simultaneous Hypotheses Testing*, has been organized by O. Teytaud et al. in May 2007.

A new criterion for graphical model learning, stressing the graph structure complexity, has been proposed in Sylvain Gelly's PhD ; the advantage of this criterion in terms of learning consistency has been demonstrated in the specific but applicatively relevant case of a small learning sample when the graph structure is not the true one.

In the context of unsupervised learning, a new latent-clustering based criterion has been proposed in the SELECT project-team, and tackled by TAO using evolutionary approaches .

Finally, AUC-like learning criteria are being considered in Arpad Rimmel's PhD, aimed at handling imbalanced classification problems with many more features than examples, motivated by chemometry applications. The dialogue with the experts in the applicative context (Accamba ANR) did not permit the assessment of the approach up to now.

The great applicative successes of Support Vector Machines
*Statistical Learning Theory*, J. Wiley, 1998.*Evolutionary Design of Geometric-Based Fuzzy Systems*. PhD thesis, Université Paris-Sud, July 2006.

In a more theoretical perspective, the feasibility of learning in higher-order logic spaces has been investigated in an average-case perspective ; a new framework has been developed to study the expected undecidability (the probability of meeting undecidable clauses) and the convergence thereof along learning.

Another way of exploring the hypothesis space is based on ensemble learning, in the hope that the whole can perform better than the sum of its parts. Along this line, a multi-objective evolutionary ensemble learning approach has been proposed, leading to some insights into how to ensure the diversity of the hypotheses along evolution or in the final population, and how to select the best ensemble .

Independently, motivated by the search for active neural cell assemblies or relevant patterns (in the context of the ACI NeuroDyne), the spatio-temporal data mining of
Magneto-Encephalographic datasets has been formalised as a multi-objective optimisation problem (finding large spatio-temporal areas with high signal correlation). An extension to
multi-objective multi-modal optimisation was required to capture the several neural cell assemblies in interplay
. Interestingly, the search for discriminant patterns among the relevant patterns turns out to be significantly
easier than directly searching for discriminant patterns

ML can simultaneously be viewed as an optimisation problem and a constraint satisfaction problem (CSP). Inspired from the Phase Transition paradigm developed in the CSP community since the
early 90s, Lorenza Saitta and Attilio Giordana have been studying relational learning after some order parameters; some implications on the limitations of existing relational learners have
been demonstrated in an early collaboration with TAO members (1999). The order parameters define a landscape, enabling to depict the average behaviour of any related algorithm through a
*Competence Map*. The comparison of the competence maps related to various
-subsumption algorithms was shown instrumental to building a meta-layer, automatically selecting the best (on average) algorithm depending on the problem instance at hand

This research direction investigates how the theoretical and algorithmic body of knowledge developed in Machine Learning can advance the fundamental study of stochastic optimisation, extend its scope and support more effective algorithms. Three types of contributions have been made; the first one is related to the theoretical study of Estimation of Distribution Algorithms; the second one considers surrogate optimisation, extending EC to deal with computationally expensive objective functions; the third one is related to Optimal Decision Making, revisiting dynamic programming and investigating multi-armed bandit algorithms.

Estimation of Distribution Algorithms (EDAs) evolve a probability distribution on the search space by repeating a series of sampling/selection/learning steps. Using Statistical Learning theory, EDAs have been studied in the context of expensive optimisation problems allowing only a small numbers of iterates , and some optimal (w.r.t. robustness) comparison-based EDAs have been proposed . Note that this research is closely related to that on surrogate models decribed in next section .

Genetic Programming (GP) extends the Evolutionary Computation paradigm to tree-structured search spaces, essentially the space of programs. In this field where theory still
lags far behind practice, several classical results of Statistical Learning theory have allowed to delineate the applicability of this technique, and to derive sufficient conditions on the
penalty term used in practice to limit the uncontrolled growth of the solution (aka
*bloat*)
*Revue d'Intelligence Artificielle*, 20(6):805–827, 2006.

More generally, several techniques borrowed from Machine Learning and Complexity Theory have been applied to theoretical investigations of Evolutionary Algorithms. This includes results on the consistency of halting criteria and sufficient conditions for convergence in non-convex settings ; the non-validity of the No-Free-Lunch Theorem in continuous optimisation ; some limitations of multi-objective optimisation without feedback from the user ; an analysis of the parametrisation of the computational effort in stochastic optimisation , ; and studies of different ways to use quasi-random points in Evolutionary Algorithms .

Evolutionary Algorithms are known to be computationally expensive. They are hence particularly concerned with what Mechanical Engineers have called Response Surface Methods, revisited in the last few years as Surrogate Models methods. The idea is to build an approximation of the objective function, and to run the optimisation algorithm (whatever the algorithm) on the approximation rather than on the original function. A first crucial issue is the choice of the approximation model. And, because the approximation has to be updated as the search proceeds, another important issue is how often this update has to be done.

Within the ANR/RNTL OMD project, TAO is in charge of the technology transfer related to surrogate methods, motivated by the expensive benchmark problems of the industrial OMD partners
(Dassault, Renault and EADS), using in particular the surrogate-based version of CMA-ES
*Proc. PPSN IX*, pp.939-948, LNCS 4193, Springer Verlag, 2006.

In some problems, the goal is to find the optimal policy in the sense of some (delayed) reward function; an intermediate step thus is to learn the value function, associating to each
problem state the reward expectation. Dynamic programming, a sound and robust approach dating back to the 50's, suffers from the curse of dimensionality

OpenDP
*NIPS Workshop on Machine Learning Open Source Software*, 2006.

Monte-Carlo planning approaches have been investigated, with the domain of computer-go as motivating application. The MoGo program
,
,
, embedding a Monte-Carlo evaluation function within the Multi-Armed Bandit framework, currently is the best
computer-go program

The Multi-Armed Bandit setting has been intensively studied in TAO from a theoretical
and applicative
perspective. The MAB extension to dynamic settings has been considered in relation with the Pascal Challenge
*Online Trading of Exploration vs Exploitation*
*Online Trading of Exploration and Exploitation Workshop, NIPS*, 2006.

One key issue in Evolutionary Computation is to design a representation and the associated variation operators, crossover and mutation, best suited to the problem at hand. In order to do so, the use of prior knowledge is highly recommended in practice. Resuming early work devoted to Structural Optimisation, the research done in TAO focuses on the design of scalable and modular representations. A promising direction in this respect is that of developmental representations.

Specifically in the domain of combinatorial optimization, all reported successes of EAs are based on the use of prior knowledge and domain-specific heuristics. An application in temporal planning have witnessed this, and further demonstrated the need for some characterisation of problem instances in order to facilitate the choice of the hyper-parameters.

Incorporating domain knowledge in Evolutionary Algorithms is mandatory after the No Free Lunch Theorem in boolean settings (and is a good idea in any other setting); the choice of the representation (and the associated variation operators, e.g. crossover and mutation) is the very first place where this can be done.

Marc Schoenauer's early work on Evolutionary Design included an original representation based on Voronoi diagrams
*Journal of Convex Analysis*, 9, pp 503–517, 2002*Concept*, 71(3):95–99, 2005.

However, this approach has many drawbacks: its lack of flexibility makes it almost impossible to address the constructibility issue. An original representation directly handling
construction plans of Lego-like structures was proposed in Alexandre Devert's PhD in 2006

The core of this developmental approach is a Neural Network, duplicated in all the 'cells' of the underlying substrata. Whereas the first work above uses standard NN evolution of topology

An on-going collaboration within the MIT-France program aims at further extending developmental systems in the context of architectural design. Ongoing work focus on applying this approach in the domain of truss structure design (phd of A. Devert) as well as evolving a developing multi-cellular system in a continuous substrate (work of N. Bredeche).

Gradually shifting toward complex system modelling, we investigated, in collaboration with the Alchemy INRIA project-team, the influence of the topology of large Neural Networks on their computational ability, with the co-supervised PhD thesis of Fei Jiang. First studies investigated the influence of topology of SOM networks on their classification performances . On-going work is concerned with Echo State Networks and their use in Engineering and Control problems.

In the meantime, on-going study on Gene Regulatory Networks (GRNs) provides yet another source of diversity for representation of (Neural) Networks: within the GENNETEC project, and
starting from Banzhaf's model of GRN
*Genetic Programming Theory and Practice*, pp 43–62, Kluwer, 2003.

The representation issue also arises for combinatorial optimisation problems, as witnessed by the new paradigm for Evolutionary Temporal Planning developped in TAO.

This original representation, based on a sequential splicing of the problem in the state space, was designed during a collaboration with Thalès
. The idea is to use a deterministic constaint programming solver (CPT

Several studies pertaining to engineering optimisation (including the work on Structural Design mentioned in Section ) are on-going, in collaboration with Renault, Dassault and EADS.

Isotherm identification in analytic chromatography, an important challenge for chemical engineers, has been considered within the ACI
*Chromalgema*. A simulation/identification platform had been designed, using Self-Adaptive Evolution Strategies and gradient based methods in a first phase. In a second phase, the use of
CMA-ES allowed to significantly improve the results ...and the usability of the platform for chemists
.

Multi-Disciplinary Optimisation, a typical application domain for Evolutionary Algorithms since several objectives (usually non-regular) are involved, has been considered in TAO. In 2007, C. LeBaron's PhD (CIFRE Renault, in its 3rd year) is interested in the global optimisation of the propulsion engine, mixing static and dynamic structural optimization with acoustic design.

Along similar lines, TAO participates in the OMD RNTL project, in charge of the general surrogate approach Work Package, and is working on one test-case provided by EADS that concerns the global optimization of a complete launcher considering structural, combustion, and trajectory planification at the same time for several possible missions (satellite positionning around the earth).

Evolutionary Robotics is another domain of application where Machine Learning and Evolutionary Computation can be usefully combined. In TAO, in the recent years, diverse controller
representations have been investigated, ranging from classical multi-layer perceptrons
*Une approche évolutionnaire de la robotique modulaire et anticipative*. PhD thesis, Université Paris-Sud, September 2005.*Evolutionary Design of Geometric-Based Fuzzy Systems*. PhD thesis, Université Paris-Sud, July 2006.

During his on-going PhD, C. Hartland first extended the use of an anticipation module to address the reality gap problem
*IEEE Intl Conf. on Robotics and Biomimetics (ROBIO)*, pp 1640 - 1645, IEEE Press, 2006.

We are also concerned with optimizing robot locomotion and morphologies. Recent works this year focused on bridging the reality gap for an evolved locomotion controler for a tetrapodal walking robot using Central Pattern Generator as a locomotion basic block. On the practical aspects, we implemented a howebrew bios update for a real world robotic kit and then ran an evolved controler on the real tetrapodal robot (in the scope of the SCOUT STIC-asie projet). We also proposed a representation formalism to deal with representing robot morphologies. The goal is to use such a representation to evolve morphologies for a given objective (typicaly a locomotion gait) (Master's thesis M2R, 2007).

Autonomic Computing, acknowledged a Grand Challenge by IBM in 2001

Autonomic Grids were considered a promising field of ML applications as Cécile Germain-Renaud provided both her expertise and extensive datasets related to the EGEE

C. Germain-Renaud, C. Loomis, J. Moscicki, and R. Texier. Scheduling for responsive grids.
*Journal of Grid Computing*, 2007. To appear.

C. Germain-Renaud, C. Loomis, R. Texier, and A. Osorio. Grid scheduling for interactive analysis. In
*HealthGrid 2006*, volume 120 of
*Studies in Health Technology and Informatics*, pp 25–33, 2006. IOS Press.

The application of EC and ML to software testing has been investigated along the EvoTest STREP (Tanja Voss, ITI, Valencia, Spain coordinator, 2006-2009), where TAO is in charge of the Evolutionary Engine at the core of the search process. The generation of test data is set as an optimisation problem. Depending on the context, the fitness can range from coverage measures (for structural testing), to CPU time or memory consumption (for functional testing of real-time embedded systems). The automatic generation of the Evolutionary Algorithm from the test objectives will be based on GUIDE (see section ), and one of the main challenges as far as Evolutionary Optimisation is concerned is to make the search engine fully autonomous, relieving the burden of adjusting evolutionary parameters by trials and errors.

Independently, in a joint work with the Software Engineering group in LRI (Marie-Claude Gaudel and Sandrine Gouraud), Nicolas Baskiotis' PhD has proposed a hybrid approach based on ML and
stochastic heuristics to overcome the drawbacks of statistical structural software testing

Preliminary studies concerning the modeling of speculative bubbles on a financial market under different rationality frameworks

Transportation systems provide other examples of social systems where interesting collective phenomena may emerge from local interactions. We have investigated such complex systems from two perspectives. In the first one we proposes a new approach of traffic reconstruction and prediction based on floating car data, by application of distributed algorithms (belief propagation) and ideas inherited from statistical physics , . In the second perspective, the mechanism of jam emergence due to variability of driver behaviours was analysed using probabilistic tools of queueing networks and statistical physics models (exclusion processes, condensation mechanisms and phase transitions) .

**Contracts managed by INRIA**

**Chromalgema**, CNRS Program ACI NIM (New Interfaces of mathematics) – 2003-2007 (14 kEur), coordinator F.James (Université d'Orléans);

participant: Anne Auger, Mohamed Jebalia, Marc Schoenauer.

**ONCE-CS**– 2005-2008 (147 kEur) European
*Coordinated Action*from FP6. Coordinator Jeff Johnson, Open University, UK;

Participants: Bertrand Chardon, Marc Schoenauer.

**OMD-RNTL**– 2005-2008 (72 kEur) Coordinator Rodolphe Leriche, Ecole des Mines de St Etienne;

Participants: Anne Auger, Olivier Teytaud and Marc Schoenauer.

**Renault**– 2005-2008 (45 kEur) side-contract to Claire LeBaron's CIFRE Ph.D.;

Participants: Claire LeBaron, Marc Schoenauer.

**EvoTest**– 2006-2009 (231 kEur) European
*Specific Targeted Research Project*from FP6. Coordinator Tanja E.J. Vos, Instituto Tecnológico de Informática, Spain;

Participants: Marc Schoenauer.

**GENNECTEC**– 2006-2009 (379 kEur) European
*Specific Targeted Research Project*from FP6. Coordinator François Képès, Génopôle and CNRS, France;

Participants: Miguel Nicolau, Marc Schoenauer.

**SCOUT**– 2007-2008 from STIC Asia program, coordinated by MICA, Hanoi (Vietnam). TAO is the INRIA correspondant for this project (15kEur). Other partners are from Vietnam (IFI,
CARGIS), China (LIAMA), Cambodia (ITC) and France (IRD, LRI-Paris Sud / TAO, MaIA-LORIA, IGN).

**Contracts managed by CNRS or Paris Sud University**

**PASCAL**, Network of Excellence, 2003-2007 (34 kE in 2005). Coordinator John Shawe-Taylor, University of Southampton. M. Sebag is manager of the Challenge Programme.

**KD-Ubiq**, Coordinated Action, 2005-2008 (19 kE). Coordinator Michael May, Fraunhofer Institute. M. Sebag is responsible of the Benchmarking WP.

**Traffic**ACI-NIM (New Interfaces of mathematics) – 2004-2007 (17,5 kEur). Coordinator Jean-Michel Loubès, Project SELECT;

Participants: M. Sebag, O. Teytaud.

**AGIR**ACI Masses de Données - 2004-2007 (30 kEur) Coordinator Cécile Germain-Renaud.

**EGEE-II**FP6 IP - 2006-2008 (10 kEur) Participants: Cécile Germain, Michèle Sebag, Xiangliang Zhang, Julien Perez.

**NeuroLog**RNTL 2007-2009 (10 kEur) Participants: Cécile Germain.

**DEMAIN**PPF (Interdisciplinary program) of the Ministry of Research 2006-2009 (10KE) Participants: Cécile Germain (coordinator), Michèle Sebag, Xiangliang Zhang, Julien Perez.

**Galileo**, Programme d'actions intégrés franco-italien – 2007 (4.2 kEur);

Participant and coordinator: Antoine Cornuéjols.

Marc Schoenauer, Board Member of ISGEC (International Society on Genetic and Evolutionary Algorithms) since 2000. This Society became the ACM SIGEVO (Special Interest Group in Evolutionary Computation) in 2006, but the board remained unchanged.

The Games Lab (University of Alberta) invited S. Gelly in 2006 and a collaboration on tree-based Monte-Carlo planning started .

Marc Schoenauer, Member of PPSN Steering Committee (Parallel Problem Solving from Nature) since 1998.

Michèle Sebag, Member of PASCAL Steering Committee (Pattern Analysis, Statistical Modeling and Computational Learning, FP6 NoE) since 2004; Member of KD-Ubiq Steering Committee (Ubiquitous Knowledge Discovery, FP6 CA) since 2006.

EGEE, Enabling Grids for E-SciencE : Cécile Germain-Renaud is a member of the NA4 steering committee, and chair for the Working Group
*Short Deadline Jobs*.

ONCE-CS, Coordinated Action, 6th Framework Program: TAO (Marc Schoenauer) is one of the main contracting nodes, responsible of WP2 - Web Portal and Services. Bertrand Chardon is paid as engineer and works on this WP.

PASCAL, Network of Excellence, 6th Framework Program: Michèle Sebag, corresponding member for Université Paris-Sud since 2003, Manager of the Challenge Programme since 2005.

Colab, ETH Zurich .

Université Lausanne .

JET, Journées Évolutionnaires Trimestrielles: Marc Schoenauer organized the first editions since their creation in 1998 until 2004. Now member of the steering Committee. A 2-days Summer School was organised in this framework at Yravals in June 2007.

Evolution Artificielle: the international conference on Evolutionary Computation, is organized in France every second year, and has acquired a world-wide reputation not only because of the good wine and food ...Marc Schoenauer is in the organizing committee since the first edition in 1994. The 8th edition took place in Tours in Ocrober 2007.

Dagstuhl Seminar: Anne Auger co- organizer for the Seminar “Theory of Evolutionary Computation” 2008.

Apprenteo, gathering the researchers of the Digiteo Lab (PCRI, CEA, SupElec, LIMSI, CMAP) now RTRA, had a second meeting on March 16th, organized by Michele Sebag.

*Evolution Artificielle* : Marc Schoenauer, founding president (1995-2004), now member of the Steering Committee. Anne Auger and Nicolas Bredeche, members of the Administrative
Committee since October 2007.

*AFIA, Association Française d'Intelligence Artificielle* : Marc Schoenauer, member of Executive (since 1998, former president (2002-2004)) ; Michèle Sebag, member of Executive
since 2000, treasurer in 2003-2004, president since 2004.

*FERA, Fédération des Equipes de Recherche en Apprentissage* : Michèle Sebag, member of the Steering Committee with Stéphane Canu, Manuel Davy and Jean-Gabriel Ganascia.

Collaborations with Remi Munos (at Ecole Polytechnique before joining the team-project SEQUEL) grounded the development of MoGo .

Vincent Vidal, Université de Lens, started to collaborate to the temporal planning application by providing the sources of his temporal planner CPT, and gradually became a full co-author of the project .

The collaboration with Gilles Celeux, Project team Select, INRIA Futurs, resulted in some original results on Latent Class Clustering .

The research on representations for the topology of large Neural Networks started after several discussions with Alchemy INRIA, about modern computing architectures.

Balàzs Kègl, from Laboratoire de l'Accélérateur Linéaire (LAL), Université Paris-Sud, is an associate member of TAO .

Collaboration with the Programming and Software Engineering group, led by Marie-Claude Gaudel, LRI, Université Paris-Sud, is about applying Machine Learning to Software Testing .

The collaboration with the young architecture group EZCT motivated the research on compact representations for Structural Design ; this collaboration also led to a joint submission to the Serousi House contestwhere we won the first price together with another candidate.

MoGo, developed in the Team by S. Gelly, J.-B. Hoock, A. Rimmel, O. Teytaud and Y. Wang, won many Kgs-tournaments (
http://

IEEE Congress on Evolutionary Computation, Sept. 25.-28 2007. Marc Schoenauer, invited plenary speaker.

ISICA'07, International Symposium on Intelligence Computation and Applications, Wuhan, Sept. 21-23. Marc Schoenauer, invited plenary speaker.

3rd Franco-Japanese Workshop, Sapporo, June 25-29, Michele Sebag, invited speaker.

First Entente Cordiale meeting, London, May 25, Michèle Sebag, invited speaker.

CMFBD07, Nancy. Complexity in learning. May, 21st, 2007. Olivier Teytaud, invited speaker.

NIPS Workshop on Machines Learning in Games, December 8, 2007. Monte-Carlo planning in the game of Go. Olivier Teytaud, Invited speaker.

Marc Schoenauer is Editor in Chief of MIT Press Evolutionary Computation Journal (since 2002)

Marc Schoenauer is Associate editor of Kluwer Genetic Programming and Evolvable Machines (since its creation in 1999), of Elsevier Theoretical Computer Science - Theory of Natural Computing (TCS-C) since its creation in 2002, of Elsevier Applied Soft Computing since its creation in 2000, of Springer Memetic Computing Journal (first issue scheduled in March 2008), and has been Associate Editor of of IEEE Transactions on Evolutionary Computation (1996-2004) and of Kluwer Journal of Heuristics (1997-2003).

Marc Schoenauer is on the Editorial Board of the book series
*Natural Computing*by Springer Verlag, and
*Mathématiques Appliquées*by SMAI (Springer-Verlag).

Michèle Sebag is member of the Editorial Board of Knowledge and Information Systems (since 2003), of Machine Learning Journal (since 2001), of Genetic Programming and Evolvable Hardware (since 2000); she has been Associate Editor of of IEEE Transactions on Evolutionary Computation (1998-2004) and of Revue d'Intelligence Artificielle (2002-2005).

Olivier Teytaud was chair of the Multiple Simultaneous Hypothesis Testing Wshop, (May, 2007, Paris).

Antoine Cornuéjols was co-chair of CAP'07 (Conférence Francophone d'Apprentissage) (July, 2007, Grenoble).

Anne Auger: Genetic and Evolutionary Computation Conference, IEEE Congress on Evolutionary Computation, Parallel Problem Solving from Nature.

Nicolas Bredèche: European Conference on Genetic Programming, ICINCO Workshop on Multi-agent Robotic Systems.

Marc Schoenauer: Genetic and Evolutionary Computation Conference, IEEE Congress on Evolutionary Computation, Parallel Problem Solving from Nature, European Conference on Genetic Programming, Evolutionary Computation for Combinatorial Optimization Problems, European Conference on Complex Systems, ...

Michèle Sebag: PC of ICML 06, 23rd International Conference on Machine Learning, ECML-PKDD 06, 17th European Conference on Machine Learning, 10th Conference on Principle and Practice of Knowledge Discovery from Databases, IJCAI 07, 20th International Conference on Artificial Intelligence; ILP, Inductive Logic Programming, PPSN, Parallel Problem Solving from Nature, EuroGP, European Conference on Genetic Programming, GECCO, Genetic and Evolutionary Computation Conference, CEC, IEEE Congress on Evolutionary Computation, ICDM, IEEE Conf. on Data Mining...

Olivier Teytaud: Approximate Dynamic Programming and Reinforcement Learning ADPRL'07

Cécile Germain: IFIP Network and Parallel Computing (NPC) since 2007; EGEE User Forum since 2007; Europar Advisory Board.

CAP, Conférence d'apprentissage: Michèle Sebag since 1999; Antoine Cornuéjols, since 1999; Olivier Teytaud since 2005.

EA, Evolution Artificielle: Marc Schoenauer and Michèle Sebag since 1994, Anne Auger, Nicolas Bredèche and Olivier Teytaud since 2005.

RFIA, Reconnaissance des Formes et IA: Michèle Sebag, member of the Editorial Committee.

*Les 40 ans de l'INRIA*, Marc Schoenauer member of the Program Committee.

Marc Schoenauer, reviewer for both ANR programs Young Researchers and Open Call (“appel blanc”); reviewer of European STREP Perplexus; Professorship evaluation for Profs K. Deb and Jon Rowe (Birmingham University, UK), and Jim Smith (UWE, Bristol, UK).

Michèle Sebag, reviewer for FP7 Strep (June 11-15, Bruxels); for both ANR programs Young Researchers and Open Call (“appel blanc”); for RNTL; member of the CNRS evaluation committee for the LINA Lab, Nantes.

Reviewer for PhD dissertation: Marc Schoenauer (3) ; Michèle Sebag (2) ;

Reviewer for Habilitation: Michèle Sebag (1)

The collection of Chairs, designed within a collaboration between TAO and with the architect consortium ECZT, have been bought in the permanent Design Collection of
Beaubourg, the French National Modern Art Museum, and exhibited from April to October 2007 (see Section
, and a more detailed desription at URL
http://

Marc Schoenauer

Lecture at the SEBASE Summer School in Birmingham in July 2007;

Two tutorials at the JET Summer School in Yravals in June 2007;

Séminaire “Complexité” organised by EuroBios in December 2007.

Michèle Sebag, Lecture at the Dagstuhl Wshop on Probabilistic and Relational Logic, May 2007.

Anne Auger: Tutorial at the JET Summer School in Yravals in Jume 2007.

Sylvain Gelly, 25/9/07, Université Paris-Sud.

Master 2 Recherche (U. Paris-Sud), Data Mining and Machine Learning (24 h): Michèle Sebag, Antoine Cornuéjols.

Master 2 Recherche (U.Paris-Sud), Artificial and Natural Perception : Nicolas Bredeche (3h).

Master 2 Recherche (U.Paris-Sud), Multi-agent Systems : Nicolas Bredeche (3h).

Master 2 Recherche (U.Paris-Sud), Artificial Evolution and Evolutionary Robotics, Anne Auger, Nicolas Bredèche and Marc Schoenauer.

Master 1 Recherche (ENS Cachan), Introduction to Machine Learning, Michèle Sebag (3h).

*Ecole Polytechnique*, Projects in Evolutionary Robotics in the
*Modex d'Electronique*: Marc Schoenauer, Cédric Hartland.

*Ecole Polytechnique*, Majeure “SEISM” (Engineering Science): one lesson (+ hands-on experiments) on Evolutionary Topological Optimum Design;

*Ecole Polytechnique*,
*Stages d'option*: Michèle Sebag, Marc Schoenauer.