- A1.3.2. Mobile distributed systems
- A1.5.2. Communicating systems
- A2.3.1. Embedded systems
- A3.4.1. Supervised learning
- A3.4.2. Unsupervised learning
- A3.4.3. Reinforcement learning
- A3.4.4. Optimization and learning
- A3.4.5. Bayesian methods
- A3.4.6. Neural networks
- A3.4.8. Deep learning
- A5.1. Human-Computer Interaction
- A5.4.2. Activity recognition
- A5.4.4. 3D and spatio-temporal reconstruction
- A5.4.5. Object tracking and motion analysis
- A5.4.6. Object localization
- A5.4.7. Visual servoing
- A5.10.2. Perception
- A5.10.3. Planning
- A5.10.4. Robot control
- A5.10.5. Robot interaction (with the environment, humans, other robots)
- A5.10.6. Swarm robotics
- A5.10.7. Learning
- A5.11.1. Human activity analysis and recognition
- A6.1.2. Stochastic Modeling
- A6.1.3. Discrete Modeling (multi-agent, people centered)
- A6.2.3. Probabilistic methods
- A6.2.6. Optimization
- A6.4.1. Deterministic control
- A6.4.2. Stochastic control
- A6.4.3. Observability and Controlability
- A6.4.6. Optimal control
- A8.2. Optimization
- A8.2.1. Operations research
- A8.2.2. Evolutionary algorithms
- A8.11. Game Theory
- A9.2. Machine learning
- A9.5. Robotics
- A9.6. Decision support
- A9.7. AI algorithmics
- A9.9. Distributed AI, Multi-agent
- A9.10. Hybrid approaches for AI
- B5.2.1. Road vehicles
- B5.6. Robotic systems
- B7.1.2. Road traffic
- B8.4. Security and personal assistance
1 Team members, visitors, external collaborators
- Ozgur Erkent [Inria, Starting Research Position, until May 2021]
- Christian Laugier [Inria, Emeritus, HDR]
- Agostino Martinelli [Inria, Researcher, from Aug 2021]
- Alessandro Renzaglia [Inria, Researcher, from Oct. 2021]
- David Sierra Gonzalez [Inria, Starting Research Position]
- Olivier Simonin [Team leader, INSA Lyon, Professor, HDR]
- Jilles Dibangoye [INSA Lyon, Associate Professor]
- Christine Solnon [INSA Lyon, Professor, HDR]
- Anne Spalanzani [Univ Grenoble Alpes, Associate Professor, HDR]
- Luc Libralesso [Université Clermont Auvergne, until Aug 2021]
- Wenqian Liu [Inria, from Nov 2021]
- Edward Beeching [UBISOFT, from Apr 2021 until Jul 2021]
- Edward Beeching [Inria]
- Alexandre Bonnefond [Inria]
- Esteban Carvalho [Univ Grenoble Alpes]
- Aurelien Delage [INSA Lyon]
- Niranjan Deshpande [Inria, until Oct 2021]
- Manuel Diaz Zapata [Inria]
- Leon Fauste [Univ Grenoble Alpes, from Oct 2021]
- Romain Fontaine [INSA Lyon]
- Jean Baptiste Horel [Inria, from Apr 2021]
- Maria Kabtoul [Inria, Until 12/2021]
- Xiao Peng [INSA Lyon]
- Mihai-Ioan Popescu [INSA Lyon, until Sep 2022]
- Manon Prédhumeau [Univ Grenoble Alpes, until Nov 2021]
- Benoit Renault [INSA Lyon, until Sept 2022]
- Loic Rouquette [INSA Lyon]
- Luiz Serafim Guardini [Renault, CIFRE]
- Samuele Zoboli [INSA Lyon]
- Albert Albert [INSA Lyon, Engineer]
- Rabbia Asghar [Inria, Engineer]
- Vincent Dufour [Inria, Engineer]
- Thomas Genevois [Inria, Engineer]
- Andres Gomez Hernandez [Inria, Engineer, until Mar 2021]
- Andres Gonzalez Moreno [Inria, Engineer]
- Vincent Le Doze [Inria, Engineer, until Sep 2021]
- Jerome Lussereau [Inria, Engineer]
- Khushdeep Singh Mann [Inria, Engineer, from Apr 2021]
- Anshul Paigwar [Inria, Engineer, until Nov 2021]
- Alessandro Renzaglia [Inria, Engineer, until Mar 2021]
- Lukas Rummelhard [Inria, Engineer]
- Alexis Saget [Inria, Engineer, until Oct 2021]
- Amrita Suresh [Inria, Engineer]
- Abhishek Tomy [Inria, Engineer, from Apr 2021]
- Stephane d'Alu [INSA Lyon, Engineer]
Interns and Apprentices
- Jerome Arjonilla [INSA Lyon, from Feb 2021 until Aug 2021]
- Hugo Duarte [INSA Lyon, from Mar 2021 until Sep 2021]
- Unmesh Patil [Inria, until Mar 2021]
- Antoine Rochebois [INSA Lyon, from Apr 2021 until Jul 2021]
- Baris Sayil [INSA Lyon, from Mar 2021 until Aug 2021]
- Pierre Susbielle [Institut polytechnique de Grenoble, from Feb 2021 until Jul 2021]
- Anouchka Ronceray [Inria]
- Ozgur Erkent [Université Hacettepe de Ankara - Turquie, from Jun 2021]
- Fabrice Jumel [École supérieure de chimie physique électronique de Lyon]
- Jacques Saraydaryan [École supérieure de chimie physique électronique de Lyon]
2 Overall objectives
2.1 Origin of the project
Chroma is a bi-localized project-team at Inria Lyon and Inria Grenoble (in Auvergne-Rhône-Alpes region). The project was launched in 2015 before it became officially an Inria project-team on December 1st, 2017. It brings together experts in perception and decision-making for mobile robotics and intelligent transport, all of them sharing common approaches that mainly relate to the field of Artificial Intelligence. It was originally founded by members of the working group on robotics at CITI lab1, led by Prof. Olivier Simonin (INSA Lyon2), and members from Inria project-team eMotion (2002-2014), led by Christian Laugier, at Inria Grenoble. Earlier members include Olivier Simonin (Prof. INSA Lyon), Christian Laugier (Inria researcher DR, Grenoble), Anne Spalanzani (Asso. Prof. HDR, Univ. Grenoble), Jilles Dibangoye (Asso. Prof. INSA Lyon) and Agostino Martinelli (Inria researcher CR, Grenoble). On January 2020, Christine Solnon (Prof. INSA Lyon) joined the team, thanks to her transfer from LIRIS lab. to CITI lab. On October 2021, Alessandro Renzaglia (Inria researcher CR, Lyon) was recruited through the Inria researcher recruitment campaign.
The overall objective of Chroma is to address fundamental and open issues that lie at the intersection of the emerging research fields called “Human Centered Robotics” 3, “Multi-Robot Systems" 4, and AI for humanity.
More precisely, our goal is to design algorithms and models that allow autonomous agents to perceive, decide, learn, and finally adapt to their environment. A focus is given to unknown and human-populated environments, where robots or vehicles have to navigate and cooperate to fulfill complex tasks.
In this context, recent advances in embedded computational power, sensor and communication technologies, and miniaturized mechatronic systems, make the required technological breakthroughs possible.
Chroma is clearly positioned in the "Artificial Intelligence and Autonomous systems" research theme of the Inria 2018-2022 Strategic Plan. More specifically we refer to the "Augmented Intelligence" challenge (connected autonomous vehicles) and to the "Human centred digital world" challenge (interactive adaptation).
2.2 Research themes
To address the mentioned challenges, we take advantage of recent advances in: probabilistic methods, machine learning, planning techniques, multi-agent decision making, and constrained optimisation tools. We also draw inspiration from other disciplines such as Sociology, to take into account human models, or Physics/Biology, to study self-organized systems.
Chroma research is organized in two main axes : i) Perception and Situation Awareness ii) Decision Making. Next, we elaborate more about these axes.
Perception and Situation Awareness.
This theme aims at understanding complex dynamic scenes, involving mobile objects and human beings, by exploiting prior knowledge and streams of perceptual data coming from various sensors.
To this end, we investigate three complementary research problems:
- Bayesian & AI based Perception: How to interpret in real-time a complex dynamic scene perceived using a set of different sensors, and how to predict the near future evolution of this dynamic scene and the related collision risks ? How to extract the semantic information and to process it for the autonomous navigation step.
- Modeling and simulation of dynamic environments: How to model or learn the behavior of dynamic agents (pedestrians, cars, cyclists...) in order to better anticipate their trajectories?
- Robust state estimation: Acquire a deep understanding on several sensor fusion problems and investigate their observability properties in the case of unknown inputs.
This theme aims to design algorithms and architectures that can achieve both scalability and quality for decision making in intelligent robotic systems and more generally for problem solving.
Our methodology builds upon advantages of three (complementary) approaches: online planning, machine learning, and NP-hard optimization problem solving.
- Online planning: In this theme we study planning algorithms for single and fleet of cooperative mobile robots when they face complex and dynamics environments, i.e. populated by humans and/or mostly unknown.
- Machine learning: We search for structural properties–e.g., uniform continuity–which enable us to design efficient planning and (deep) reinforcement learning methods to solving complex single or multi-agent decision-making tasks.
- Offline constrained optimisation problem: We design and study approaches based on Constraint Programming (CP) and meta-heuristics to solve NP-hard problems such as planning and routing problems, for example.
Chroma is also concerned with applications and transfer of the scientific results. Our main applications include autonomous and connected vehicles, service robotics, exploration & mapping tasks with ground and aerial robots. Chroma is currently involved in several projects in collaboration with automobile companies (Renault, Toyota) and some startups (see Section 4).
The team has its own robotic platforms to support the experimentation activity (see5). In Grenoble, we have two experimental vehicles equipped with various sensors: a Toyota Lexus and a Renault Zoe; the Zoe car has been automated in December 2016. We have also developed two experimental test tracks respectively at Inria Grenoble (including connected roadside sensors and a controlled dummy pedestrian) and at IRT Nanoelec & CEA Grenoble (including a road intersection with traffic lights and several urban road equipments). In Lyon, we have a fleet of UAVs (Unmanned Aerial Vehicles) composed of 4 PX4 Vision, 2 IntelAero and 5 mini-UAVs Crazyflies. We have also a fleet of ground robots composed of 16 Turtlebot and 3 humanoids Pepper. The platforms are maintained and developed by contractual engineers, and by Lukas Rummelhard, from SED, in Grenoble.
3 Research program
The Chroma team aims to deal with different issues of autonomous navigation and intelligent transport: perception, decision-making and cooperation. Figure 1 schemes the different themes and sub-themes investigated by Chroma.
We present here after our approaches to address these different themes of research, and how they combine altogether to contribute to the general problem of robot navigation. Chroma pays particular attention to the problem of autonomous navigation in highly dynamic environments populated by humans and cooperation in multi-robot systems. We share this goal with other major robotic laboratories/teams in the world, such as Autonomous Systems Lab at ETH Zurich, Robotic Embedded Systems Laboratory at USC, KIT 6 (Prof. Christoph Stiller lab and Prof. Ruediger Dillmann lab), UC Berkeley, Vislab Parma (Prof. Alberto Broggi), and iCeiRA 7 laboratory in Taipei, to cite a few. Chroma collaborates at various levels (visits, postdocs, research projects, common publications, etc.) with most of these laboratories.
3.2 Perception and Situation Awareness
The team carries out research across the full (software) stack of an autonomous driving system. This goes from perception of the environment to motion forecasting and trajectory optimization. While most of the techniques that we develop can stand on their own and are typically tested on our experimental platforms, we slowly move towards their complete integration. This will result in a functioning autonomous driving software stack. Beyond that point, we envision to focus our research on developing new methods that improve the individual performance of each of the stack’s components, as well as the global performance. This is alike to how autonomous driving companies operate (e.g. Waymo, Cruise, Tesla). The research topics that we explore are addressed by a very long list of international research laboratories and private companies. However, in terms of scope, we find the research groups of Prof. Dr.-Ing. Christoph Stiller (KIT), Prof. Marcelo Ang (NUS), Prof. Daniela Rus (MIT) and Prof. Wolfram Burgard (TRI California) to be the closest to us; we have active interconnections with these research groups (e.g. in the scope of the IEEE RAS Technical Committee AGV-ITS we are co-chairing any of the related annual Workshops). At the national level, we are cooperating in the framework of several R&D projects with UTC, Institut Pascal Clermont-Ferrand, University Gustave Eiffel, and the Inria Teams RITS, Acentauri and Convecs. Our main research originality relies in the combination of Bayesian approaches, AI technologies, Human-Vehicle interactions models, Robust State Estimation formalism, and a mix of Real experiments and Formal methods for validating our technologies in complex real world dynamic scenarios.
Our research on visual-inertial sensor fusion is closely related to the research carried out by the group of Prof. Stergios Roumeliotis (University of Minnesota) and the group of Prof. Davide Scaramuzza (University of Zurich). Our originality in this context is the introduction of closed-form solutions instead of filter-based approaches.
1. Bayesian & AI based Environment Perception and Forecasting
In order to be able to operate in a dynamic environment surrounded by humans or other robots, an autonomous system needs to Perceive, Understand and Forecast any changes of its surroundings. We have investigated the perception task from different angles, most of them with the shared nexus of involving both Bayesian approaches and Deep Learning models.
Bayesian Perception: The team's work on Bayesian approaches has given rise over the last decade to numerous publications and to some patents and licensed software8. Over the past 4 years, we have focused on collision risk assessment, safe navigation and semantic processing issues, with a strong emphasis on the integration and the validation of our technologies on real-time experimental platforms. The new results gave rise to the filing of two patents and to a transfer of technology with a French SME and with a Japanese international company (R&D work still ongoing). The main advances relate to (1) the generalization of the principle of Bayesian fusion in order to be able to integrate connected sensors located outside the ego-vehicle (infrastructure or other vehicles), (2) new real-time methods allowing to better estimate the risks of collision and make navigation safer (including path planning), and (3) experimental validation principle combining real tests, synchronized simulated environments (augmented reality principle) and formal methods 50, 80, 92, 49. A new PhD thesis addressing the subject of validating AI-based perception components in autonomous vehicles, with a particular focus on formal validation methods, has started in 2021 in cooperation with the Inria Convecs team (funding: French project PRISSMA).
Situation awareness & Environment Forecasting: In addition to the above-mentioned work, we have developed AI-based methods for extracting the semantic information of observed scenes 78, 77, 79, 105. In particular, we have developed hybrid methods for 3D object detection 94, 10824, semantic mapping of the environment from a top-view 66, 67, 68 and ground plane estimation and segmentation 93. We have likewise investigated the use of domain adaptation techniques to mitigate the degradation of some of the previous algorithms across different weather conditions 65, 62 and sensor modalities 105. In terms of environment forecasting, two PhD theses were dedicated to the task of predicting the future behavior of surrounding traffic 48, 109. The first of them targeted road intersections; the second focused on highway scenarios and leveraged a behavioral model learned from driving demonstrations 106 to cope with the long-term prediction horizon necessary for highways. Both theses also explored how to integrate the forecasting models into a behavioral planning framework 107. In addition to these forecasting models, we have also carried out research on localization 47, tracking 20, detection of abnormal behaviors 118, and contextualization of complex risk mitigation scenarios 104. A new PhD thesis addressing the subject of “Hybrid Sensor fusion” was launched in 2021 (funding: EU CPS4EU).
2. Modeling and simulation of dynamic environments
Modeling pedestrian's behaviors with a limited amount of data is a challenge. In Pavan Vasishta's PhD and the ANR Valet project, we proposed to learn typical pedestrian behavior using HMM, starting from prior knowledge itself derived from the JJ Gibson’s sociological principles of “Natural Vision” and “Natural Movement”. The work assumes that there are some parts of the observed scene which are more attractive to pedestrians and some areas, repulsive 113.
Pedestrians simulation. We studied the behavior of pedestrians in a shared space with an autonomous vehicle by modeling and simulating realistic pedestrian behavior. Our approach integrates empirical observations and concepts from social science into an agent-based model and simulator for an application in robotics. At each step, the proposed model has been evaluated and validated through simulations of many scenarios and comparisons with real-world data. Then we proposed the simulator SPACiSS, "Simulator for Pedestrians and an Autonomous Car in Shared Spaces", an open source simulator based on the PedSim framework that can simulate interactions between pedestrians and vehicles in different shared space scenarios. With the integration in the ROS framework, commonly used in robotics, the software is designed as an environment to test autonomous navigation systems 95, 9627, 14.
Robots in human flows. Another work, led by J. Saraydaryan, was conduced on simulating pedestrians and mobile robots in indoor environments, by extending PedSim. This work aimed to simulated scenarios where robots perceive and map human flows and compute paths by exploiting such information (in the cost-function of A* algorithm) 10272.
3. Robust State Estimation
Our investigations have been in the framework of the following two fundamental problems: (i) the visual and inertial sensor fusion problem, and (ii) the unknown input observability problem. Regarding the former, our activity was the follow-up of our previous research on this domain, published in 88, 81, and 75. We extended the above results to the cooperative case (in particular, we studied the case of two agents). First, we proved the equivalence between the cooperative visual-inertial sensor fusion problem and a Polynomial Equation System (PES) 87. The power of this equivalence is twofold. From one side, it allowed us to obtain its analytical solution 86. In other words, we obtained the closed-form solution of this important problem and, as a result, we were able to determine the state without the need of an initialization. The analytical solution was obtained by using the method based on the Macaulay resultant matrices developed by the AROMATH team at INRIA Sophia Antipolis 111. This solution, that is the exact solution of the cooperative visual and inertial sensor fusion problem in the case of two agents, was published in 86. From another side, the theoretical study of the PES, allowed us to obtain fundamental insights into all the structural properties of the problem 82. The usefulness of this analysis has also been illustrated with simulations.
Regarding the second problem, i.e., the unknown input observability problem, we provided a fundamental new result. This problem was introduced and firstly investigated during the sixties and has remained unsolved for 50 years. During the last five years, we have obtained the analytic solution of this fundamental open problem. Very surprisingly, the complexity of this solution is comparable to the complexity of the standard method to check the state observability in the case without unknown inputs (i.e., the observability rank condition). On the other hand, the analytic derivations and all the proofs necessary to analytically derive the new solution and its convergence properties and to prove its general validity are very complex and they are extensively based on the Ricci calculus with tensors. The analytic criterion becomes much easier for driftless systems and in the presence of a single unknown input. The solution of this simple case (driftless systems and in the presence of a single unknown input) is given in 84. Due to its complexity, the general solution was published in a book 85.
- Main long-term industrial collaborations: IRT Nanoelec & CEA 10 years), Toyota Motor Europe (12 years, R&D center Zaventen-Brussels), Renault (10 years), SME Easymile (3 years), Sumitomo Japan (started in 2019), Iveco (3 years in the framework of STAR project). These collaborations are funded either by Industry or by National, Regional or European projects. The main outputs are common publications, softwares development, patents or technological transfers. Two former doctoral students of our team were recruited some years ago by respectively TME Zaventen and Renault R&D centers.
- Main academic collaborations: Institut Pascal Clermont-Ferrand, University Compiegne (UTC), University Gustave Eiffel, several Inria Teams (RITS, Acentauri, Convecs, Rainbow). These collaborations are most of the time conducted in the scope of various National R&D projects (FUI, PIA, ANR, etc.). The main outputs are scientific and software exchanges. In addition, we had a fruitful collaboration with B. Mourrain from the Inria-Aromath team. The goal was to find the analytical solution of a polynomial equation system that fully characterizes the cooperative visual-inertial sensor fusion problem with 2 agents.
- International cooperation for scientific exchanges and IEEE scientific events organization, e.g. NUS & NTU Singapore, Peking University, TRI Mountain View, KIT Karlsruhe, Coimbra University. We have constantly interacted with Prof. D. Scaramuzza from the university of Zurich to make our findings in visual inertial sensor fusion more usable in a realistic context (e.g., for the autonomous navigation of drones).
3.3 Decision Making
In his reference book Planning algorithms76, S. LaValle discusses the different dimensions that make the motion-planning a complex problem, which are the number of robots, the obstacle regions, the uncertainty of perception and action, and the allowable velocities. In particular, it is emphasized that multiple robot planning in complex environments are NP-hard problems which implies that exact approaches have exponential time complexities in the worst case (unless P=NP) Moreover, dynamic and uncertain environments, as human-populated ones, expand this complexity. In this context, we aim at scaling up decision-making in human-populated environments and in multi-robot systems, while dealing with the intrinsic limits of the robots and machines (computation capacity, limited communication). To address these challenges, we explore new algorithms and AI architectures following three directions: online (or real-time) planning, machine learning to adapt to the complexity of uncertain environments, and combinatorial optimization to deal with offline constrained problems. Combining these approaches, seeking to scale up them, and also evaluating them with real platforms are also elements of originality of our work.
We share these goals with other laboratories/teams in the world, such as the Mobile Robotics Laboratory and Intelligent Systems "IntRoLab" at Sherbrooke University (Montreal) led by Prof. F. Michaud, the Learning Agents Research Group (LARG) within the AI Lab at the University of Texas (Austin) led by Prof. P. Stone, the Autonomous Systems Lab at ETH Zurich (Switzerland) led by R. Siegwart, the Robotic Sensor Networks Lab at the University of Minnesota (USA) led by Prof. V. Isler, and the Autonomous Robots Lab at NTNU (Norway) to cite a few. At Inria, we share some of the objectives with the LARSEN team in Nancy (multi-agent machine learning, multi-robot planning) and the RAINBOW team in Rennes (multi-UAV planning). We have also collaborations with the ACENTAURY team, in Sophia Antipolis, about autonomous navigation among humans. In France, among other labs involved on similar subjects, we can cite the LAAS (CNRS, Toulouse), the ISIR lab (CNRS, Paris Sorbonne Univ.) and the MAD team in Caen University. In the more generic domain of problem solving, we can mention the TASC team at LS2N (Nantes), the CRIL lab (Lens) and, at the international level, we can cite the Insight Centre for Data Analytics (Cork, Ireland).
1. Online Planning
In this theme, we address online planning when robots/vehicles/UAVs have to navigate or work in complex and generally unknown environments. The challenge comes from the necessity for entities to decide actions in real time and with limited information. Hereafter we present our main results following different problem contexts.
Social and ethic navigation
We investigated new methods to build safe and socially compliant trajectories among humans using models of behaviors presented in Axis I. A focus on navigation of an autonomous vehicle among crowds has been done were there is no free space to plan a safe trajectory (M. Kabtoul's PhD, ANR Hianic Project). We proposed a solution that is a proactive social navigation framework. The system is based on the idea of the coupled navigation behavior between the pedestrian and the vehicle in shared spaces. The system takes into account the cooperative nature of human behavior and exploits it to explore new navigation options, and navigate the shared space “proactively" 74, 73. We obtained promising results using the SPACISS simulator previously described. We also investigated ethic navigation when an autonomous vehicle cannot avoid a collision with an object or a vulnerable road user (L. Serafim Guardini's PhD). We exploit accidentology data where each class of object or agent presents an injury probability with respect to the impact speed and ethical/economical/political factors. Our method generates a cost map containing a collision probability along with its associated risk of injury, which are used to plan trajectories with the lowest risk 104.
We also investigated another navigation problem, which is NAMO (Navigation Among Movable Obstacles). We extended NAMO in two directions considering social constraints when robots move obstacles (B. Renault's PhD). First, we examined where obstacles should be optimally moved with regards to space access (called Social-NAMO). We derived new spatial cost functions and NAMO algorithms preserving accesses 97. Second, we generalized the problem to robots (called MR-NAMO). We proposed a local coordination strategy exploiting Social-NAMO rearrangement properties and offering online efficient plans (under review). For evaluation, we developped a simulator (S-NAMO-SIM) that we share with the communauty.
An important part of our research is focused on designing new online planning solutions for multi-robot systems to accomplish exploration and observation gathering tasks in complex and dynamic environments. Problems related to computational and communication constraints are here crucial, and distributed approaches are fundamental to overcome them and obtain robust solutions.
Following these goals, a particular attention has been given to the exploration/mapping of 3D environments with a team of aerial vehicles, for which we presented a new decentralized solution based on the combination of a stochastic optimization approach with a more classic strategy that exploits frontier points 100. In 99, we then showed how this approach is part of a more general formulation that can deal with the optimal deployment coverage as well, for which we also studied the impact of an offline initialization step based on very partial information on the environment. Stochastic-based planning methods have been also exploited to propose a new source seeking strategy 98. This solution is based on robust estimations of the signal gradient obtained by the robots while exploring the environment in predefined symmetric formations 56 and that are then used to bias a correlated random walk that guides the robots.
Connected to this subject, we also investigated multi-robot strategies to cover complex scenes/ environments, by introducing multi-resolution maps. This concerned human activity observation by a set of ground robots (89) and more recently inspection of large structures with aerial robots (in the framework of the European project BugWright2, in collaboration with LSL, Austria).
In the context of self-reorganized systems, or swarm robotics, we explore how collective navigation can be defined and adapted to fleets of communicating robots. In A. Bonnefond's PhD, we studied flocking models, where each agent communicates locally its speed and position, and by simulating radio propagation we examined how obstacles impact the flocking robustness. Then we extended and combined two standard models, Olfati-Saber and Vásárhelyi, to improve their ability to stay connected while evolving in environments with different obstacle distributions 17. This work is supported by the Inria/DGA Dynaflock project and by the national Equipex+ TIRREX infrastructure. Related to this work, we address Controled Mobility where navigation must take into account the need of maintaining the fleet connectivity. In R. Grunblatt's PhD, we proposed a distributed algorithm that optimizes communication in a swarm of UAVs by changing their antenna orientation online 13. This work is continued in the ANR CONCERTO project.
2. Machine Learning
Machine learning techniques for decision-making are autonomous algorithms for computing behaviors, also called policies or strategies, which describe what action an agent should take when at a given state. Learning to act in complex environments requires knowledge about the structure of underlying decision-making problems. We contribute in revealing insightful underlying structures, exploiting them within existing or customized algorithms, and showing they improve performances in a wide variety of decision-making problems. For instance, optimal policies may have concise representations of their mental states, e.g., sufficient statistics, ego-centric neural-network architectures.
In the context of POMDPs9, dec-POMDPs, zs-POSGs10, and st-POSGs, we have identified a unified framework for solving these problems 64. Our methodological framework splits into five steps. First, we have shown that all these problems are convertible into a single problem called a common-knowledge game 64. Then, we recast the (possibly discrete) problem into a continuous one. Furthermore, we establish uniform continuity properties for general and hierarchical dec-POMDPs. We proved that optimal value functions for general and hierarchical dec-POMDPs are PWLC 64, 117. We also established optimal value functions for general zs-POSGs that are Lipschitz continuous 57, 58. Finally, we designed planning algorithms for games against Nature 69, pure competitive games 12, as well as for pure cooperative games 64. We extended exact and deep reinforcement learning algorithms for general dec-POMDPs 64. We finally conducted applications of deep reinforcement along two main directions: autonomous transportation systems 55; and automatic control systems 30.
Creating agents capable of high-level reasoning based on structured memory was the main topic of C. Wolf during its Inria "delegation" in Chroma (2017-19). In the PhD thesis of E. Beeching we proposed EgoMap, a spatially structured metric neural memory architecture in deep reinforcement learning. Through visualizations, the agent learns to map relevant objects in its spatial memory without any supervision purely from reward, see 53 (ICPR), 52 (ECCV) and 51 (ECML). In late 2019, C. Wolf obtained the AI Chair "REMEMBER", involving O. Simonin and J. Dibangoye from Chroma and L. Matignon from LIRIS lab/Univ Lyon 1. In this context, the PhD of P. Marza proposes to use an implicit representation of the environment to navigate towards targets. Neural fields are a recent category of approaches where the geometry, and eventually the semantics, of a 3D scene are encapsulated into the weights of a trained neural network.
As mentioned in Axis I, making appropriate decisions for safe robot / vehicle navigation is highly dependent on the results of the perception step. Indeed, the critical elements of a complex dynamic scene must first be extracted, classified and tracked in real time, by combining Bayesian and AI approaches to build "semantic grids" 3637, 66, 65. Two types of decision-making based on RL were developed for autonomous driving on highways and road junctions respectively, as part of the doctoral theses of David Sierra-Gonzalez 109 and Mathieu Barbier 48.
A last activity concerns our involvement in the RoboCup11 competition. In 2017 we founded the LyonTech team12 to participate to the RoboCup@Home challenge within the Pepper league. We worked on learning and integrating high-level functionalities such as object and people recognition, mapping and planning in indoor environments (see 71). We reached the 5th place in 2018, the 3rd place in 2019, the 2d place in 2021, and awarded the best scientific paper at the RoboCup conference in 2019 103.
3. Offline constrained optimisation problems
In this theme, we study and design algorithms for solving NP-hard constrained optimization problems (COPs) such as vehicle routing problems, or multi-agent path finding problems. We more particularly study approaches based on Constraint Programming (CP), that provides declarative languages for describing these problems by means of constraints, and generic algorithms for solving them by means of constraint propagation.
In the context of the European project BugWright2 and the PhD thesis of Xiao Peng (started in Nov. 2020), we study a new multi-agent path finding problem for tethered robots, that are attached with cables to anchor points. In 26, we show how to compute bounds for this problem by solving well known assignment problems, and we introduce a CP model for solving the problem to optimality. In 91, we extend this approach to the case of non point-sized robots.
In the context of the PhD thesis of Romain Fontaine (started in Oct. 2020), we study vehicle routing problems with time-dependent cost functions (such that travel times between locations to visit may change along the day), and we design new solving approaches based on an anytime extension of A* 70. We study the impact of time-window constraints on the satisfiability and the difficulty of routing problems in 28.
Besides planning and routing problems which are at the core of Chroma topics, we also investigate the interest of using CP solvers for solving cryptanalysis problems 23.
- Social navigation: collaboration with Julie Dugdale and Dominique Vaufreydaz, Asso. Prof. at LIG Laboratory and with Prof. Philippe Martinet from Acentori Inria team (common PhD students and projects: ANR VALET and ANR HIANIC).
- Multi-robot exploration: Prof. Cedric Pradalier from GeorgiaTech Metz/CNRS (France) leading the European project BugWright2 — Prof. Luca Schenato from University of Padova (Italy).
- Swarm of UAVs: Prof. Isabelle Guérin-Lassous from LIP lab/Inria DANTE team (common PhD students and projects: ANR CONCERTO, INRIA/DGA DynaFlock) — Research Director Isabelle Fantoni from CNRS/LS2N Lab in Nantes (common projects: ANR CONCERTO, Equipex TIRREX) — Dr. Melanie Schranz, senior researcher at LakeSide Lab, Austria (BugWright2 project).
- Machine Learning: strong collaborations with Christian Wolf, Naver Labs Europe & LIRIS lab/INSA13 (common PhD students and projects: ANR DELICIO, ANR AI Chair REMEMBER) and with Laetitia Matignon, Asso. Prof. at LIRIS lab/Univ Lyon 1 (common projects and Phd/Master students) — Akshat Kumar, Asso. Prof. at School of Computing and Information Systems (Singapore) — Abdallah Saffidine, Research Asso. at University of New South Wales (Sydney) — Prof. Stéphane Canu at INSA Rouen — Olivier Buffet, Vincent Thomas and François Charpillet, Inria researchers in LARSEN team, Nancy (common publications and project ANR PLASMA).
- Semantic grids for autonomous navigation: Collaboration with Toyota Motor Europe (TME), R&D center Zaventen (Brussels). R&D contracts, common publications and patents.
- Constrained optimization: Omar Rifki (Post-doc) and Thierry Garaix (Ass. Prof) at LIMOS Ecole des Mines de Saint Etienne — Pascal Lafourcade and François Delobel, Asso. Prof. at LIMOS Université Clermont-Ferrand.
4 Application domains
Applications in Chroma are organized in two main domains : i) Future cars and transportation of persons and goods in cities, ii) Service robotics with ground and aerial robots. These domains correspond to the experimental fields initiated in Grenoble (eMotion team) and in Lyon (CITI lab). However, the scientific objectives described in the previous sections are intended to apply equally to these applicative domains. Even if our work on Bayesian Perception is today applied to the intelligent vehicle domain, we aim to generalize to any mobile robots. The same remark applies to the work on multi-agent decision making. We aim to apply algorithms to any fleet of mobile robots (service robots, connected vehicles, UAVs). This is the philosophy of the team since its creation.
4.2 Future cars and transportation systems
Thanks to the introduction of new sensor and ICT technologies in cars and in mass transportation systems, and also to the pressure of economical and security requirements of our modern society, this application domain is quickly changing. Various technologies are currently developed by both research and industrial laboratories. These technologies are progressively arriving at maturity, as it is witnessed by the results of large scale experiments and challenges such as the Google’s car project and several future products announcements made by the car industry. Moreover, the legal issue starts to be addressed in USA (see for instance the recent laws in Nevada and in California authorizing autonomous vehicles on roads) and in several other countries (including France).
In this context, we are interested in the development of ADAS 14 systems aimed at improving comfort and safety of the cars users (e.g., ACC, emergency braking, danger warnings), of Fully Autonomous Driving functions for controlling the displacements of private or public vehicles in some particular driving situations and/or in some equipped areas (e.g., automated car parks or captive fleets in downtown centers or private sites), and of Intelligent Transport including optimization of existing transportation solutions.
Since about 8 years, we are collaborating with Toyota and with Renault-Nissan on these applications (bilateral contracts, PhD Theses, shared patents), but also recently with Volvo group (2016-20). We are also strongly involved (since 2012) in the innovation project Perfect then now Security for autonomous vehicle of the IRT 15 Nanoelec (transportation domain). Since 2016, we have been awarded two European projects (the "ENABLE" H2020 ECSEL project 16 and the "CPS4EU" H2020 ECSEL project) involving major European automotive constructors and car suppliers. In these projects, Chroma is focusing on the embedded perception component (models and algorithms, including the certification issue). Chroma (A. Spalanzani) led also the ANR Hianic (2018-21) dealing with pedestrian-vehicle interaction for a safe navigation.
In this context, Chroma has two experimental vehicles equipped with various sensors (a Toyota Lexus and a Renault Zoe. , see. Fig. 2) , which are maintained by Inria-SED 17 and that allow the team to carry out experiments in realistic traffic conditions (Urban, road and highway environments). The Zoe car has been automated in December 2016, through our collaboration with the team of P. Martinet (IRCCyN Lab, Nantes), that allow new experiments in the team.
4.3 Service robotics with ground and aerial robots
Service robotics is an application domain quickly emerging, and more and more industrial companies (e.g., IS-Robotics, Samsung, LG) are now commercializing service and intervention robotics products such as vacuum cleaner robots, drones for civil applications, entertainment robots, etc. One of the main challenges is to propose robots which are sufficiently robust and autonomous, easily usable by non-specialists, and marked at a reasonable cost. We are involved in developing fleet of ground robots and aerial ones, see Fig. 2. Since 2016, we study solutions for 3D observation/exploration of complex scenes or environments with a fleet of UAVs (eg. BugWright2 H2020 project, Dynaflock Inria/DGA project) or ground robots (COMODYS FIL project 89).
A more recent challenge for the coming decade is to develop robotized systems for assisting elderly and/or disabled people. In the continuity of our work in the IPL PAL 18, we aim to propose smart technologies to assist electric wheelchair users in their displacements and also to control autonomous cars in human crowds (see Figure 8 for illustration). This concerns our recent "Hianic" ANR project. Another emerging application is humanoid robots helping humans at their home or work. In this context, we address the problem of NAMO (Navigation Among Movable Obstacles) in indoor environments (see PhD of B. Renault started on 2018). More generally we address navigation and object search with humanoids robots such as Pepper, in the context of the RoboCup-Social League competition.
5 Social and environmental responsibility
5.1 Footprint of research activities
- Due to the Covid, the vast majority of our trips for PhD defenses and for selection committees were replaced by videoconferences in 2021. While there is no question of systematizing the use of videoconferences, we changed our habits to offer a real alternative and seek a better compromise between environmental impact and user-friendliness.
- Some of our research topics involve high CPU usage. Some researchers in the team use the Grid5000 computing cluster, which guarantees efficient management of computing resources and facilitates the reproducibility of experiments. Some researchers in the team keep a log summarizing all the uses of computing servers in order to be able to make statistics on these uses and assess their environmental footprint more finely.
5.2 Impact of research results
- Romain Fontaine's thesis aims to develop algorithms for the efficient optimization of delivery rounds in the city. Our work is motivated by taking into account environmental constraints, set out in the report "Assurer le fret dans un monde fini" from The Shift Project (theshiftproject.org/wp-content/uploads/2022/03/Fret_rapport-final_ShiftProject_PTEF.pdf)
- Léon Fauste's thesis focuses on the design of a decision support tool to choose the scale when relocating production activities (work in collaboration with the Inria STEEP team). This resulted in a publication in collaboration with STEEP at ROADEF 32 and the joint organization of a workshop on the theme "Does techno-solutionism have a future?" during the Archipel conference which will be organized in June 2022 (archipel.inria.fr/).
- Some of us are strongly involved in the development of INSA Lyon training courses to integrate DDRS issues (Christine Solnon is a member of the CoPil and the GTT on environmental and social digital issues). This participation feeds our reflections on the subject.
6 Highlights of the year
- Manon Prédhumeau (PhD stud.) won the best student paper of the AAMAS 2021 conf. 27
- Pierre Marza (PhD stud.) won the MultiON Challenge at CVPR2021 Embodied AI Workshop19
- Christian Laugier has been elevated to the grade of IEEE Senior Member in August 2021 (nomination and support of three recognized scientists).
6.2 New projects
- ANR "MaestrIoT" (Multi-Agent Trust Decision Process for the Internet of Things). Started on Dec. 2021.
- Grand Defi "PRISSMA" (Plateforme de Recherche et d'Investissement pour la Sûreté et la Sécurité de la Mobilité Autonome). Started in April 2021.
- ANR "Annapolis" (AutoNomous Navigation Among Personal mObiLity devIceS). Will start in 2022 (project.inria.fr/annapolis/)
7 New software and platforms
7.1 New software
Robotics, Environment perception
CMCDOT is a Bayesian filtering system for dynamic occupation grids, allowing parallel estimation of occupation probabilities for each cell of a grid, inference of velocities, prediction of the risk of collision and association of cells belonging to the same dynamic object. Last generation of a suite of Bayesian filtering methods developed in the Inria eMotion team, then in the Inria Chroma team (BOF, HSBOF, ...), it integrates the management of hybrid sampling methods (classical occupancy grids for static parts, particle sets for parts dynamics) into a Bayesian unified programming formalism , while incorporating elements resembling the Dempster-Shafer theory (state "unknown", allowing a focus of computing resources). It also offers a projection system of the estimated scene in the near future, to reference potential collisions with the ego-vehicle or any other element of the environment, as well as very low cost pre-segmentation of coherent dynamic spaces (taking into account speeds). It takes as input instantaneous occupation grids generated by sensor models for different sources, the system is composed of a ROS package, to manage the connectivity of I / O, which encapsulates the core of the embedded and optimized application on GPU Nvidia (Cuda), allowing real-time analysis of the direct environment on embedded boards (Tegra X1, X2). ROS (Robot Operating System) is a set of open source tools to develop software for robotics. Developed in an automotive setting, these techniques can be exploited in all areas of mobile robotics, and are particularly suited to highly dynamic and uncertain environment management (e.g. urban scenario, with pedestrians, cyclists, cars, buses, etc.).
Amaury Nègre, Lukas Rummelhard, Jean-Alix David, Christian Laugier
7.1.2 Ground Elevation and Occupancy Grid Estimator (GEOG - Estimator)
Robotics, Environment perception
GEOG-Estimator is a system of joint estimation of the shape of the ground, in the form of a Bayesian network of constrained elevation nodes, and the ground-obstacle classification of a pointcloud. Starting from an unclassified 3D pointcloud, it consists of a set of expectation-maximization methods computed in parallel on the network of elevation nodes, integrating the constraints of spatial continuity as well as the influence of 3D points, classified as ground-based or obstacles. Once the ground model is generated, the system can then construct a occupation grid, taking into account the classification of 3D points, and the actual height of these impacts. Mainly used with lidars (Velodyne64, Quanergy M8, IBEO Lux), the approach can be generalized to any type of sensor providing 3D pointclouds. On the other hand, in the case of lidars, free space information between the source and the 3D point can be integrated into the construction of the grid, as well as the height at which the laser passes through the area (taking into account the height of the laser in the sensor model). The areas of application of the system spread across all areas of mobile robotics, it is particularly suitable for unknown environments. GEOG-Estimator was originally developed to allow optimal integration of 3D sensors in systems using 2D occupancy grids, taking into account the orientation of sensors, and indefinite forms of grounds. The ground model generated can be used directly, whether for mapping or as a pre-calculation step for methods of obstacle recognition or classification. Designed to be effective (real-time) in the context of embedded applications, the entire system is implemented on Nvidia graphics card (in Cuda), and optimized for Tegra X2 embedded boards. To ease interconnections with the sensor outputs and other perception modules, the system is implemented using ROS (Robot Operating System), a set of opensource tools for robotics.
Amaury Nègre, Lukas Rummelhard, Lukas Rummelhard, Jean-Alix David, Christian Laugier
7.1.3 Zoe Simulation
Simulation of INRIA's Renault Zoe in Gazebo environment
This simulation represents the Renault Zoe vehicle considering the realistic physical phenomena (friction, sliding, inertia, ...). The simulated vehicle embeds sensors similar to the ones of the actual vehicle. They provide measurement data under the same format. Moreover the software input/output are identical to the vehicle's. Therefore any program executed on the vehicle can be used with the simulation and reciprocally.
Christian Laugier, Nicolas Turro, Thomas Genevois
7.1.4 Hybrid-state E*
Path planning with Hybrid-state E*
Considering a vehicle with the kinematic constraints of a car and an environment which is represented by a probabilistic occupancy grid, this software produces a path from the initial position of the vehicle to its destination. The computed path may include, if necessary, complex maneuvers. However the suggested path is often the simpler and the shorter.
This software is designed to take benefit from bayesian occupancy grids such as the ones computed by the CMCDOT software.
Christian Laugier, Thomas Genevois
Simulator, Multi-agent, Crowd simulation, Autonomous Cars, Pedestrian
These ROS packages are useful to support robotic developments that require the simulation of pedestrians and an autonomous vehicle in various shared spaces scenarios. They allow: 1. in simulation, to pre-test autonomous vehicle navigation algorithms in various crowd scenarios, 2. in real crowds, to help online prediction of pedestrian trajectories around the autonomous vehicle.
Individual pedestrian model in shared space (perception, distraction, personal space, pedestrians standing, trip purpose). Model of pedestrians in social groups (couples, friends, colleagues, family). Autonomous car model. Pedestrian-autonomous car interaction model. Definition of shared space scenarios: 3 environments (business zone, campus, city centre) and 8 crowd configurations.
Simulation of pedestrians and an autonomous vehicle in various shared space scenarios. Adaptation of the original Pedsim_ros model to simulate heterogeneous crowds in shared spaces (individuals, social groups, etc.). The car model is integrated into the simulator and the interactions between pedestrians and the autonomous vehicle are modeled. The autonomous vehicle can be controlled from inside the simulation or from outside the simulator by ROS commands.
Manon Prédhumeau, Anne Spalanzani, Julie Dugdale, Lyuba Mancheva
Simulation, Navigation, Robotics, Planning
2D Simulator of NAMO algorithms (Navigation Among Movable Obstacles) ROS compatible
Simultion of UAV fleets with Gazebo/ROS
The simulator includes the following functionality : 1) Simulation of the mechanical behavior of an Unmanned Aerial Vehicle : * Modeling of the body's aerodynamics with lift, drag and moment * Modeling of rotors' aerodynamics using the forces and moments' expressions from Philppe Martin's and Erwan Salaün's 2010 IEEE Conference on Robotics and Automation paper "The True Role of Accelerometer Feedback in Quadrotor Control". 2) Gives groundtruth informations : * Positions in East-North-Up reference frame * Linear velocity in East-North-Up and Front-Left-Up reference frames * Linear acceleration in East-North-Up and Front-Left-Up reference frames * Orientation from East-North-Up reference frame to Front-Left-Up reference frame (Quaternions) * Angular velocity of Front-Left-Up reference frame expressed in Front-Left-Up reference frame. 3) Simulation of the following sensors : * Inertial Measurement Unit with 9DoF (Accelerometer + Gyroscope + Orientation) * Barometer using an ISA model for the troposphere (valid up to 11km above Mean Sea Level) * Magnetometer with the earth magnetic field declination * GPS Antenna with a geodesic map projection.
Vincent Le Doze
Vincent Le Doze
Insa de Lyon
Swarm Protocol And Navigation Kontrol
Communication and distance measurement in an uav swarm
7.2 New platforms
Participants: Olivier Simonin, Alessandro Renzaglia, Vincent Dufour, Alexandre Bonnefond.
- PX4 Vision UAV platform : the team acquired 4 quadrirotor UAVs during year 2021. This platform is funded and used in the projects "Dynaflock" (Inria-DGA) and ANR "CONCERTO".
8 New results
8.1 Bayesian Perception
Participants: Christian Laugier, Lukas Rummelhard (SED), Andres Gonzalez Moreno, Jerome Lussereau, Thomas Genevois, Alessandro Renzaglia, Nicolas Turro (SED), Amrita Suresh, Rabbia Asghar.
Recognized as one of the core technologies developed within the team over the years (see related sections in previous activity report of Chroma, and previously e-Motion reports), the CMCDOT framework is a generic Bayesian Perception framework, designed to estimate a dense representation of dynamic environments and the associated risks of collision, by fusing and filtering multi-sensor data. This whole perception system has been developed, implemented and tested on embedded devices, incorporating over time new key modules. In 2021, this framework, and the corresponding software, has continued to be the core of many important industrial partnerships and academic contributions, and to be the subject of important developments, both in terms of research and engineering. Some of those recent evolutions are detailed below.
8.1.1 Extension of the CMCDOT framework to 3D Semantic Occupancy Grids
Participants: Andres Gonzalez Moreno, Lukas Rummelhard, Christian Laugier.
The CMCDOT perception framework has been reworked and extended:
- To better fuse data from different sensors, of different type and with different view points, probabilistic fusion of occupancy grids have been extended in 3 dimensions.
- To enrich perception and prediction capabilities, the whole framework now represents, fuses and filters semantic states instead of sole occupancy.
- To observe such semantic states in 3D grids, methods to generate probabilistic estimates in 3D grids from semantic estimation from neural networks in camera views, have been developed
These framework evolutions led to important software evolutions, including the development of numerous new software bricks, software factorizations and class abstractions, preparing for future upgrades. The first prototype has been developed and is being tested in simulation and with real data, with promising results.
8.1.2 Cooperative Perception with Dynamic Occupancy Grids
Participants: Andres Gonzalez Moreno, Rabbia Asghar, Jerome Lussereau, Nicolas Turro (SED), Lukas Rummelhard, Christian Laugier.
In order to expand coverage and overall perception capabilities of an embedded perception system, data exchange between sensing agents is a valuable solution. If occupancy grids are a well-suited tool to fuse data from heterogeneous sources with different viewpoints, the mere size of these structures makes their efficient communication a real challenge. On an experimental plateform consisting of a infrastructure unit equipped with 2 lidars, a Nvidia TX2 card executing the CMCDOT framework and VTX capabilities, and a connected autonomous Zoe plateform (with its own sensors and CMCDOT framework running), preliminary work on such communication has been conducted. Two main methods of grid transmission from the infrastructure to the autonomous vehicle have been developed and tested:
- a first version which extracts mobile objects from the dynamic occupancy grid of the infrastructure, sends object information to the vehicle using VTX communication, then reconstructs the corresponding occupancy grid based on these (taking into account added uncertainties) and fuses it with local perception.
- a second version which directly compresses the infrastructure occupancy grid, sends it via UDP, and fuses it with the vehicle grid.
Raising many technical challenges, both methods provided promising results, leading to further works next year.
8.1.3 Integration of environment motion prediction in the collision avoidance system
Participants: Thomas Genevois, Christian Laugier.
For several years, CHROMA has been developing navigation and collision avoidance software (DWA Planner), and applied it in various applications (ZOE navigation, ADAS emergency avoidance system, navigation of the MoovIT project). Among its latest developments, GPU optimizations allowed for collision risk estimation module improvements.
The DWA Planner samples possible trajectories over a short time horizon (approximately 2500 trajectories, each projected over a horizon of 6 seconds) and estimates the probable positions of the controlled vehicle according to a control-command uncertainty model. The occupancy grid is then read at the given positions to estimate the collision risks.
A new module has been developed to extrapolate the occupancy grid from the estimated motion of cell from the CMCDOT. Each moving occupancy particle is projected across multiple cells, representing the occupancy at the next instants. An uncertainty model in acceleration and steering is applied.
By reading the predicted occupancy grids, the DWA can anticipate the movements of other actors to plan trajectories.
8.1.4 Augmented Reality tools for advanced automotive software testing
Participants: Thomas Genevois, Christian Laugier.
CHROMA has also developed an Augmented Reality framework for testing and validation of software on the Renault Zoe experimental vehicle. This framework provides a flexible way to introduce any virtual element in real time in the data of the LiDAR sensors of the vehicle. Our Augmented Reality accurately handles all possible occlusions between real and virtual elements. The representability of test scenes generated by the augmented reality framework has been experimentally proven. It is then possible to easily and safely place the whole vehicle and all its software, from perception to control, in hybrid but realistic test scenes. This new testing methodology is intended to be a bridge between Vehicle-in-the-Loop and real-world testing.
8.1.5 Development of an industrial AIV, light perception and navigation systems
Participants: Thomas Genevois, Amrita Suresh, Christian Laugier.
CHROMA has been working with the company AKEOPLUS, in a new use case, in the scope of the regional R&D project Booster MoovIT, to continue the development of the embedded version of perception and navigation software. This project involves developing a versatile AIV (Autonomous Intelligent Vehicle) dedicated to the logistics of warehouses and industrial workshops. While AKEOPLUS develops the mechanics, the electronics, the basic automatisms of the robot as well as the supervision system and the user interface, CHROMA develops the functions of perception, path planning and local navigation. The jointly developed AIV should be tested in the workshop of a third-party company, Spanninga. If this specific use case implies new developments, these will also benefit the other applications of the software suite built around the CMCDOT. Such developments included:
- multi-instance CMCDOT for multimodal perception: a new grid-based data fusion method, instancing different CMCDOT for each sensor modality, then fusion the filtered grids, offering promising results
- Bayesian-Occupancy-Grid-based planning without kinematic constraints: a simplified version of the Zoe-adapted Hybrid-State-E* previously-developed method, allowing in these simplified circumstances for quicker computations. Also include added functionalities, like path-smoothing and a partial-replanning service
- Adaptation of the navigation software interface: to allow for management by a supervision system, including multiple online parameters, options, error and status messages, etc.
8.1.6 Validation of AI-based algorithms in autonomous vehicles
Participants: Jean-Baptiste Horel, Alessandro Renzaglia, Anshul Paigwar, Radu Mateescu (Convecs), Christian Laugier.
In the last years, there has been an increasing demand for regulating and validating intelligent vehicle systems to ensure their correct functioning and build public trust in their use. Yet, the analysis of safety and reliability poses a significant challenge. More and more solutions for autonomous driving are based on complex AI-based algorithms whose validation is particularly challenging to achieve. An important part of our work has been recently devoted to tackle this problem, finding new suitable approaches to validate probabilistic algorithms for perception and decision making in autonomous driving, and investigating how simulations and experiments in controlled environment can help to solve this challenge. This activity, started with our participation in the European project Enable-S3 (2016-2019), is now continuing with the PRISSMA project, started in April 2021, where we participate in collaboration the the Inria Convecs team. This project, funded by the French government in the framework of the “Grand Défi: Sécuriser, certifier et fiabiliser les systèmes fondés sur l’intelligence artificielle”, regroups several companies and public institutes in France and tackles the challenge of the validation and certification of artificial intelligence based solutions for autonomous mobility. A new PhD thesis (J.-B. Horel), co-supervised between Chroma and Convecs teams, started in April 2021 in the framework of PRISSMA.
Within this context, our main work in 2021 has been focused on the automatic generation of large numbers of relevant critical scenarios for autonomous driving simulators 21. This approach is based on the generation of behavioral conformance tests from a formal model (specifying the ground truth configuration with the range of vehicle behaviors) and a test purpose (specifying the critical feature to focus on). The obtained abstract test cases cover, by construction, all possible executions exercising a given feature, and can be automatically translated into the inputs of autonomous driving simulators. We illustrate our approach by generating thousands of behavior trees for the CARLA simulator for several realistic configurations. An extension of this work, where we model check the traces of the simulation runs, combining the advantages of temporal logic verification and quantitative analyses to validate a probabilistic collision risk estimation, is currently ongoing.
8.2 Situation Awareness & Decision-making for Autonomous Vehicles
Participants: Ozgur Erkent, David Sierra-González, Anshul Paigwar, Christian Laugier, Manuel Alejandro Diaz-Zapata, Andrés Gómez H., Alessandro Renzaglia, Jilles Dibangoye, Luiz Serafim-Guardini, Anne Spalanzani, Wenqian Liu.
In this section, we present all the novel results in the domains of perception, motion prediction and decision-making for autonomous vehicles.
8.2.1 Situation Understanding and Motion Forecasting
Participants: Özgür Erkent, David Sierra-González, Wenqian Liu, Christian Laugier.
In the context of our cooperation with Toyota Motor Europe, we study new approaches to understand the environment around an autonomous vehicle. More particularly, we focus on generating semantic grid representations of the environment by fusing information from multiple sensors (e.g. RGB cameras and LIDAR), and on tracking and forecasting the behavior of any detected dynamic obstacles.
Our main output in 2021 is a Multiple Object Detection and Tracking algorithm that works in the above-mentioned semantic grid representation of the environment 20. In this work, an object detector detects the objects 108 and a Grid Track Network (GTN) takes the information from occupancy 101 and semantic 68 grids, and combines it to track and detect the objects in bird's eye view. The results are evaluated on the Kitti MOT dataset and show comparable results with the state-of-the-art.
In terms of motion forecasting, we have developed in 2021 a novel method to forecast the lane intentions of surrounding vehicles using Graph Neural Networks. This work is currently pending for publication.
Future work will further explore deep learning sensor-fusion techniques for semantic grid generation.
8.2.2 Fusing Event-based and RGB camera for Robust Object Detection in Adverse Conditions
Participants: Abhishek Tomy, Anshul Paigwar, Khushdeep Singh Mann, Alessandro Renzaglia, Christian Laugier.
The ability to detect objects, under image corruptions and under different weather conditions is vital for deep learning models especially when applied to real-world applications such as autonomous driving. Traditional RGB-based detection fails under these conditions and it is thus important to design a sensor suite that is redundant to failures of the primary frame-based detection. Event-based cameras can complement frame-based cameras in low-light conditions and high dynamic range scenarios that an autonomous vehicle can encounter during navigation. Accordingly, we propose a redundant sensor fusion model of event-based and frame-based cameras that is robust to common image corruptions. The method utilizes a voxel grid representation for events as input and proposes a two-parallel feature extractor network for frames and events. Our sensor fusion approach is more robust to corruptions by over 30% compared to only frame-based detections and outperforms the only event-based detection. The model is trained and evaluated on the publicly released DSEC dataset.
This work has been accepted for presentation at the 2022 IEEE International Conference on Robotics and Automation (ICRA), Philadelphia, USA 29.
8.2.3 Frustum-PointPillars: A Multi-Stage Approach for 3D Object Detection using RGB Camera and LiDAR
Participants: Anshul Paigwar, David Sierra-González, Özgür Erkent, Christian Laugier.
A better understanding of the objects in 3D facilitates better decision-making and path planning. RGB Cameras and LiDAR are the most commonly used sensors in autonomous vehicles for environment perception. Many approaches have shown promising results for 2D detection with RGB Images, but efficiently localizing small objects like pedestrians in the 3D point cloud of large scenes has remained a challenging area of research. We propose a novel method, Frustum-PointPillars, for 3D object detection using LiDAR data. Instead of solely relying on point cloud features, we leverage the mature field of 2D object detection to reduce the search space in the 3D space. Then, we use the Pillar Feature Encoding network for object localization in the reduced point cloud. On the KITTI test set our method outperforms other multi-sensor SOTA approaches for 3D pedestrian localization (Bird’s Eye View) and ranked among the top 5 at the time of submission while achieving a significantly faster runtime of 14 Hz.
This work has been published and presented at the 2021 IEEE/CVF International Conference on Computer Vision (ICCVW) Workshop on Autonomous Vehicle Vision 24.
8.2.4 STADIE-Net: Stagewise Disparity Estimation from Stereo Event-based Cameras
Participants: Abhishek Tomy, Anshul Paigwar, Alessandro Renzaglia, Christian Laugier.
Event-based cameras complement the frame-based cameras in low-light conditions and high dynamic range scenarios that a robot can encounter for scene understanding and navigation. Apart from that, the comparatively cheaper cost in relation to a LiDAR sensor makes this a viable candidate when designing a sensor suite for a robot designed to operate in a dynamic environment. One of the challenges that the sensor suite needs to address is the ability to provide a 3D scene understanding of the environment that would enable the robot to localize obstacles in the environment. This work evaluates the accuracy with which an event-based camera can support this task by providing the disparity estimate between the left and right camera frames which can be utilized to calculate the depth of surroundings. A new deep network architecture, named STADIE-Net is proposed that takes advantage of stagewise refinement and prediction of disparity using events from 2 neuromorphic cameras in a stereo setup. The method utilizes voxel grid representation for events as input and proposes a 4 stage network going from coarse to finer disparity prediction. The model is trained and evaluated on the publicly released DSEC dataset that has data recorded from multiple cities using event-based and frame-based cameras mounted on a moving vehicle. Experimental results show comparable accuracy with the baseline method provided for the DSEC dataset.
This work was presented at Conference on Computer Vision and Pattern Recognition, Workshop on Event-based Vision (CVPRW), 2021, Nashville, United States 45.
8.2.5 Predicting Future Occupancy Grids in Dynamic Environment with Spatio-Temporal Learning
Participants: Khushdeep Singh, Abhishek Tomy, Anshul Paigwar, Alessandro Renzaglia, Christian Laugier.
Reliably predicting future occupancy of highly dynamic urban environments is an important precursor for safe autonomous navigation. Common challenges in the prediction include forecasting the relative position of other vehicles, modeling the dynamics of vehicles subjected to different traffic conditions, and vanishing surrounding objects. To tackle these challenges, we propose a Spatio-temporal prediction network pipeline that takes the past information from the environment and semantic labels separately for generating future occupancy predictions. Compared to the current SOTA, our approach predicts occupancy for a longer horizon of 3 seconds and in a relatively complex environment from the nuScenes dataset. Our experimental results demonstrate the ability of Spatio-temporal networks to understand scene dynamics without the need for HD-Maps and explicit modeling of dynamic objects. We publicly release our occupancy grid dataset based on nuScenes to support further research.
This work has been submitted for review at the 2022 IEEE Intelligent Vehicle Symposium (IV).
8.2.6 Panoptic Grid generation from monocular camera images
Participants: Manuel Alejandro Diaz-Zapata, Özgür Erkent, David Sierra-Gonzalez, Nicolas Turro, Lukas Rummelhard, Jilles Dibangoye, Christian Laugier.
During the year 2021, given our work on segmentation of camera images 19 we continued the work towards the panoptic segmentation of occupancy grids given camera images. As a first approach, we used information from a stereo pair to perform panoptic segmentation given the corresponding orthogonal projection of the RGB image based on the estimated depth. With the RGB information in the grid space, a modified SOLOv2 116 network was used to perform both instance and semantic segmentation on the projected images. This network was first trained on a mix of the projection of the ground truth labels available in the Cityscapes Dataset 61 and a small subset of densely annotated samples from it.
As future work, we plan to use other datasets such as the NuScenes dataset 59 and move to a sensor fusion approach where LiDAR and camera images can be used together to exploit their corresponding features for a better generation of semantic grids.
8.2.7 Real-time Collision Risk Estimation based on Stochastic Reachability Spaces
Participants: Unmesh Patil, Anshul Paigwar, Alessandro Renzaglia, Christian Laugier.
To guarantee the safety of autonomous vehicles and the robustness of driver assistance systems, it is essential to have a reliable evaluation of the risk of collision with the traffic participants. To estimate the risk, the system needs to perceive the environment, track the objects and predict their future actions. However, all the uncertainties involved in both sensing and prediction make this problem particularly challenging. In this work 25, we studied new probabilistic models to obtain stochastic reachability spaces for vehicles and pedestrians detected in the scene. We then exploit these probabilistic predictions of the road-users' future positions, along with the expected ego-vehicle trajectory, to estimate the probability of collision risk in real-time. The proposed stochastic models only depend on the velocity, acceleration, tracked bounding box, and the class of the detected object. This information can easily be obtained through off-the-shelf 3D object detection frameworks. As a result, the proposed approach for collision risk estimation is widely applicable to a variety of autonomous vehicle platforms. To validate our approach, we tested the stochastic motion prediction on the KITTI dataset. Further experiments were also carried out in the CARLA simulator, by reproducing realistic collision scenarios, with the goal of demonstrating the effectiveness of our risk prediction.
8.3 Robust state estimation (Sensor fusion)
Participants: Agostino Martinelli.
This research is the follow up of Agostino Martinelli's investigations carried out during the last seven years, which mainly consisted in the derivation of the analytical solution of the unknown input observability problem. This is the problem of obtaining a simple analytical criterion to check the observability of the state that characterizes a nonlinear system when its dynamics are driven by inputs which are unknown and act as disturbances. This was an open problem, formulated during the 1960's by the automatic control community and that remained unsolved for half century.
In December 2018, Martinelli was invited by the Society for Industrial and Applied Mathematics (SIAM) to write a book with the solution of this problem. This was the main work during 2019 85. The book contains a general analytical solution of this open problem. The solution is an algorithm that automatically provides the observability of a given system (technically speaking it provides the observability codistribution), independently of its complexity and nonlinearity, and in the presence of unknown inputs. On the other hand, this solution is based on an important assumption that, in the book, was called canonicity with respect to the unknown inputs. In addition, even in the case of systems that satisfy this assumption, the convergence criterion of the algorithm provided in 85 can fail in some special cases.
The research carried out during 2021 has been focused on the following three main topics:
- Obtaining a general convergence criterion of the algorithm in the canonical case.
- Dealing with systems that do not satisfy the canonicity assumption.
- Investigating the problem of unknown input reconstruction, which is strongly related to the problem of state observability in the presence of unknown inputs.
All these objectives have been successfully accomplished and, at the moment, published on arXiv 83. The same results have been accepted for publication, in the format of a very long paper, by the Journal of Information Fusion and, part of these results, are also conditionally accepted for publication on the Transaction on Automatic Control.
We exhaustively dealt with the systems that do not belong to the category of the systems that are canonical with respect to their unknown inputs. We provided new definitions of this concept and we obtained a more general solution that can also be applied to systems that are not canonical with respect to their unknown inputs. In addition, we showed that, in some cases, we can transform a non canonical system into a canonical system. We called these systems canonizable. The new proposed solution can deal with the case where the system is not canonical and not even canonizable. This new solution is also provided in the form of a new algorithm. Finally, as a simple consequence of these results, we also provided the answer to the problem of unknown input reconstruction which is intimately related to the problem of state observability.
8.4 Motion-planning in dense pedestrian environments
Under the coordination of Anne Splanazani, we study new motion planning algorithms to allow robots/ vehicles to navigate in human populated environment while predicting pedestrians' motions. We model pedestrians and crowd behaviors using notions of social forces and cooperative behavior estimations. We investigate new methods to build safe and socially compliant trajectories using theses models of behaviors. We propose proactive navigation solutions as well as deep learning ones. The works of year 2021 are presented below.
8.4.1 Modelling crowds and autonomous vehicles using Extended Social Force Models
Participants: Manon Predhumeau (LIG Grenoble & Chroma), Anne Spalanzani, Julie Dugdale (LIG Grenoble).
The focus of this work has been on the realistic simulation of crowds in shared spaces with an autonomous vehicle (AV), see Fig. 8. We proposed an agent-based model for pedestrian reactions to an AV in a shared space, based on empirical studies and the state of the art. The model includes an AV with Renault Zoé car’s characteristics, and pedestrians’ reactions to the vehicle. The model extends the Social Force Model with a new decision model, which integrates various observed reactions of pedestrians and pedestrians groups. We performed a qualitative and quantitative evaluation of the proposed model, through comparisons of the simulated trajectories with ground truth trajectories. The proposed model reproduces well the various pedestrians behaviors: running to cross, stopping to wait without deviating, staying in social group to avoid the AV. The model can be used for predicting pedestrians’ displacement, linear velocity, orientation, and approach distance around the AV in fundamental interactions (AAMAS'21 27).
We implemented this model using Pedsim_ros and proposed an open source simulation tool to help the robotic community with AV developments. The simulator integrates various observed pedestrians behaviours, with individual pedestrians as well as pedestrian groups with four different types of relationships (SCS'20 96). Complex shared spaces situations have been modeled, which require the interaction of a large number of pedestrians responding to the AV movement. The simulated AV can be controlled by an external ROS module, so that AV navigation algorithms can be tested in various realistic, crowded shared space environments, before real-world experiments. Moreover, simulations can reproduce real world scenes and can run faster than real-time to help live prediction of pedestrian trajectories around the AV.
8.4.2 Proactive Navigation for navigating dense human populated environments
Participants: Maria Kabtoul, Anne Spalanzani, Philippe Martinet [Chorale, Inria Sophia Antipolis].
Developing autonomous vehicles capable of navigating safely and socially around pedestrians is a major challenge in intelligent transportation. This challenge cannot be met without understanding pedestrians’ behavioral response to an autonomous vehicle, and the task of building a clear and quantitative description of the pedestrian to vehicle interaction remains a key milestone in autonomous navigation research. As a step towards safe proactive navigation in a space shared with pedestrians, we started to introduce in 2018 a pedestrian-vehicle interaction behavioral model. The model estimates the pedestrian’s cooperation with the vehicle in an interaction scenario by a quantitative time-varying function. Using this cooperation estimation the pedestrian’s trajectory is predicted by a cooperation-based trajectory planning model (see Fig. 9.a). Both parts of the model are tested and validated using real-life recorded scenarios of pedestrian-vehicle interaction. The model is capable of describing and predicting agents’ behaviors when interacting with a vehicle in both lateral and frontal crossing scenarios. In early 2019, we used this cooperation based behavioral model to design a proactive longitudinal velocity controller. The suggested velocity control strategy enabled the vehicle to cooperate and proactively interact with its surrounding rather than merely reacting to the behavior of the pedestrians. This resulted in more natural driving patterns, avoiding the sub-optimal navigation solutions which result in over-reactive methods. The method is tested and validated in lateral interactions with a flow of pedestrians. The method showed a high gain in the time to reach the goal in comparison with reactive navigation, while ensuring the pedestrians safety. In 2020, we integrated the previous navigation system with the PedSim simulator under ROS. Next, we worked on a proactive and human-like maneuvering system to deploy the steering control part of the navigation system. We adopted a two-step approach. In the first step, the navigation space is explored proactively and an optimal navigation channel is found using a fuzzy logic model. This proactive dynamic channel is guaranteed to exist even in dense scenarios where obstacle free paths do not exist. In the second step, a human-like steering is adapted using a Quintic transition path and a sliding mode control approach. The resulting system is tested under ROS in combination with the longitudinal velocity controller and the cooperation-based behavioral model. The resulting system is validated in both frontal and lateral interactions with dense crowds (see Fig. 9.b). The system results in major advantage in the travel time avoiding the freezing of the vehicle, while ensuring the pedestrian’s safety. This work has been published in ITSC'20 73, ICRA'20 74 and ICRA'22 22
8.5 Online planning and learning for robot navigation
8.5.1 Deep-Learning for task-based motion planning
Participants: Edward Beeching, Pierre Marza (PhD. student LIRIS), Christian Wolf (LIRIS/INSA Lyon), Jilles Dibangoye, Olivier Simonin, Laetitia Matignon (LIRIS/Lyon 1).
Our goal is the automatic learning of robot navigation in complex environments based on specific tasks and from visual input. The robot automatically navigates in the environment in order to solve a specific problem, which can be posed explicitly and be encoded in the algorithm (e.g. find all occurences of a given object in the environment, or recognize the current activities of all the actors in this environment) or which can be given in an encoded form as additional input, like text. Addressing these problems requires competences in computer vision, machine learning and AI, and robotics (navigation and paths planning).
A critical part for solving these kind of problems involving autonomous agents is handling memory and planning. Many control problems in partially observed 3D environments involve long term dependencies and planning. Solving these problems requires agents to learn several key capacities: spatial reasoning (exploring the environment and learning spatio-temporal regularities) and semantic mapping (discovering semantics from interactions). While solutions exist for semantic mapping and semantic SLAM 60, 110, a more interesting problem arises when the semantics of objects and their affordances are not supervised, but defined through the task and thus learned from reward.
PhD thesis of Edward Beeching (since 2018, Inria grant): After showing that through visualizations an agent can learn to map relevant objects in its spatial memory without any supervision purely from reward (see Fig. 10.a and 53, 52, 51) we proposed a 3D training framework. E. Beeching developed the Godot RL Agents, which is an interface between DeepRL algorithms and the Godot Game Engine (Linietsky 2014). This new framework allows to create 2D & 3D games to learn behaviors with automated testing and multi-process parallel training (see Fig. 10.b). In the fall of 2021, E. Beeching wrote his doctoral thesis, which he will defend on May 3, 2022.
PhD thesis of Pierre Marza (since Dec. 2020). In the context of the REMEMBER AI Chair, P. Marza started a PhD on the subject "Large-scale learning of high-level navigation skills for autonomous agents in 3D environments" (supervised by C. Wolf, L. Matignon and O. Simonin). We propose to extend the advances to the sequential decision-making of agents in a context of planning and control in complex 3D environments. A key question is the ability to learn representations with minimal human intervention and annotation. From the Egocentric Map approach, we introduced supplementary supervision in the form of auxiliary tasks designed to favor the emergence of spatial perception capabilities in agents trained for a goal-reaching downstream objective. This allowed us to win the MultiON Challenge at CVPR2021 Embodied AI Workshop. A paper presenting our architecture with the auxiliary tasks is under review.
8.5.2 Navigation Among Movable Obstacles: extension to multi-robot and social constraints
Participants: Benoit Renault, Olivier Simonin, Jacques Saraydaryan, Fabrice Jumel.
In 2018 we started to study the NAMO planning problem (Navigation Among Movable Obstacles). Since 2019, the PhD thesis of Benoit Renault aims to extend NAMO in two directions:
- Defining Social-NAMO, by considering where obstacles should be optimally moved with regards to space access. We have defined new spatial cost functions and extended existing NAMO algorithms with the ability to maintain area accesses (connectivity) for humans and robots, see IROS publication 97. We also developed a simulator for NAMO, and its extensions, called S-NAMO-SIM (see Fig. 11.a).
- Defining Multi-robot NAMO (MR-NAMO). It consists in generalizing the problem to robots that can move the objects of the environment in order to reach their individual goal positions. We have proposed a first coordination strategy based on local rules. We have shown that considering the social cost we introduced with S-NAMO offers a significant gain for solving MR-NAMO because of the rearranging it provides (see Fig. 11.b).
In 2021, a particular effort has been made to create varied and complex multi-robot NAMO scenarios. This allowed to extend the evaluation of the proposed approaches. In addition, we identifed the different deadlocks that can occurs between robots. We then proposed some local rules (heuristics) allowing to solve certain deadlocks, by exploiting the social cost map computed in S-NAMO (a conference paper is in preparation). Since the end of 2021, B. Renault is writting his doctoral thesis .
8.6 Swarm of UAVs: control, optimization and communication
8.6.1 Multi-UAV Exploration and Visual Coverage of 3D Environments
Participants: Alessandro Renzaglia, Olivier Simonin, Jilles Dibangoye, Vincent Dufour.
Multi-robot teams, especially when involving aerial vehicles (UAVs 20), are extremely efficient systems to help humans in acquiring information on large and complex environments. In these scenarios, cooperative coverage and exploration are two fundamental problems with numerous important applications, such as mapping, inspection of complex structures, and search and rescue missions. In both cases, the robots' goal is to navigate through the environment and cooperate to maximize the observed area, either by finding optimal static configurations that provides the best global view in the case of the coverage or by maximizing the new observed areas at every step until the environment becomes completely known in the case of the exploration.
Following our recent results on these problems 100, 99, in the last year we focused on a strictly related path planning problem where a fleet of UAVs must use both prior information and online gathered data to efficiently inspect large surfaces such as ship hulls and water tanks (work submitted to IROS22, currently under review.). This work is conducted in the framework of the H2020 European project BugWright2, where one central use-case is focused on corrosion detection and inspection on cargo ships. In our case, UAVs can detect corrosion patches and other defects on the surface from low-resolution images. If defects are detected, they get closer to the surface for a high-resolution inspection. The prior information provides expected defects locations and is affected by both false positives and false negatives. The mission objective is to prioritize the close-up inspection of defected areas while keeping a reasonable time for the coverage of the entire surface. In this work, we propose two solutions to this problem: a coverage algorithm that divides the problem into a set of Traveling Salesman Problems (Part-TSP) and a cooperative frontier approach that introduces frontier utilities to incorporate the prior information (Coop-Frontier). We extensively tested our solutions in simulation and compare them with alternative solutions. Additional work on this subject, which involves a Master internship currently ongoing, aims at leveraging the probabilistic prior information to optimally partition the environment and equally distribute the expected inspection time among the robots.
8.6.2 Controlled mobility & communication optimization in swarm of UAVs
Participants: Remy Grunblatt [PhD. student LIP & CITI], Isabelle Guerin-Lassous [Inria/Lyon 1 Dante team], Olivier Simonin.
At the same time Remy Grunblatt defended his PhD on beginning of 2021 38, 13, we started the CONCERTO ANR ("mobilité Contrôlée et cOmmunicatioNs effiCaces dans une flottE de dRones auTo-Organisée"). The first year was dedicated to build a state of the art on controlled mobility of swarm of drones, to discover partner experimental platforms, to select a common UAV platform (the PX4 Vision) and to define a common scenario/model to study and experiment. We chosen to study a leader-follower flocking model (based on the Olfati-Saber model) that we aim at measuring and controling the communication flows in outdoor deployment. As a first stage, we prepared indoor experiments in Nantes LS2N platform.
8.6.3 Decentralized control of swarm of UAVs: Flocking models
Participants: Alexandre Bonnefond [Phd. Inria/CITI],, Isabelle Guerrin-Lassous [LIP/Inria/Dante Lyon 1],, Olivier Simonin, Vincent Dufour.
This work is conduced through the Inria/DGA "AI" project "DYNAFLOCK" (2019-22). It funds the PhD thesis of Alexandre Bonnefond.
In this work, we study existing flocking models (decentralized swarm navigation) and propose extensions to improve their abilities to deal with environments having obstacles impacting the communication quality. Often depicted as robust systems, there is yet a lack of understanding how flocking models compare and how they are impacted by the communication quality when they exchange control data. In 2020, we first modeled the obstacles’ impact on communications in a simulator that we use to optimize flocking parameters (see. Fig. 12 a,b). Second, we extended and combined two standard models, Olfati-Saber 90 and Vásárhelyi et al. 112 (see Fig. 12 b,c), to improve their ability to stay connected while evolving in environments with different obstacles distributions. This work has been published in IROS 2021 17 and JFSMA 2021 31 conferences.
In 2021, the work continued in two directions : 1) by simulating more accuratly these models in the Gazebo 3D simulator, in order to prepare real experiments with the PX4 Vision platfom, 2) by modeling the agent's pressure inside the flock. From this new individual measure we derived an asymetric repulsion law, which allows to reduce agent collisions and the fragmentation of the flock. A publication is in preparation.
8.7 Multi-Agent Sequential Decision-Making under Uncertainty
This research is the follow up of a subgroup led by Jilles S. Dibangoye carried out during the last five years, which include foundations of sequential decision making by a group of cooperative or competitive robots or more generally artificial agents. To this end, we explore combinatorial, convex optimization and reinforcement learning methods.
8.7.1 Optimally solving zero-sum games using centralized planning for decentralized control theory
Participants: Aurelien Delage [PhD Student], Jilles S. Dibangoye, Olivier Buffet [Inria Nancy, Larsen], Vincent Thomas [Inria Nancy, Larsen], Abdallah Saffidine [Univ. New South Whales], Christopher Amato [Univ. New Hampshire], François Charpillet [Inria Nancy, Larsen].
During the last two years, we investigated deep and standard reinforcement learning for solving systems with multiple agents and different information structures. Our preliminary results include:
- (Theoretical) – As an extension of 63 in the competitive cases, we characterize the optimal solution of two-player fully and partially observable stochastic games.
- (Theoretical) – We further exhibit new underlying structures of the optimal solution for both non-cooperative two-player settings with information asymmetry, one agent sees what the other does and sees.
- (Algorithmic) – We extend a non-trivial procedure for computing such optimal solutions.
This work aims at reinforcing a recent theory and algorithms to optimally solving a two-person zero-sum POSGs (zs-POSGs). That is, a general framework for modeling and solving two-person zero-sum games (zs-Games) with imperfect information. Our theory builds upon a proof that the original problem is reducible to a zs-Game—but now with perfect information. In this form, we show that the dynamic programming theory applies. In particular, we extended Bellman equations 54 for zs-POSGs, and coined them maximin (resp. minimax) equations. Even more importantly, we demonstrated Von Neumann & Morgenstern’s minimax theorem 115114 holds in zs-POSGs. We further proved that value functions—solutions of maximin (resp. minimax) equations—yield special structures. More specifically, the optimal value functions are Lipchitz-continuous. Together these findings allow us to extend planning techniques from simpler settings to zs-POSGs. To cope with high-dimensional settings, we also investigated low-dimensional (possibly non-convex) representations of the approximations of the optimal value function. In that direction, we extended algorithms that apply for convex value functions to lipschitz value functions. This research direction lead to many publications in different avenues 58 along with technical reports 12.
PhD thesis of Aurelien Delage (since Sept. 2020, ANR JCJC PLASMA). This thesis aims at extending ideas conducted on cooperative multi-agent partially observable systems to pure competitive ones.
8.7.2 HSVI fo zs-POSGs using Concavity, Convexity and Lipschitz Properties
Participants: Delage Aurelien [PhD Student], Jilles S. Dibangoye, Olivier Buffet [Inria Nancy, Larsen].
Dynamic programming and heuristic search are at the core of state-of-the-art solvers for sequential decision-making problems. In partially observable or collaborative settings (e.g., POMDPs and Dec-POMDPs), this requires introducing an appropriate statistic that induces a fully observable problem as well as bounding (convex) approximators of the optimal value function. This approach has succeeded in some subclasses of 2-player zero-sum partially observable stochastic games (zs-POSGs) as well, but failed in the general case despite known concavity and convexity properties, which only led to heuristic algorithms with poor convergence guarantees. We overcome this issue, leveraging on these properties to derive bounding approximators and efficient update and selection operators, before deriving a prototypical solver inspired by HSVI that provably converges to an -optimal solution in finite time, and which we empirically evaluate. This opens the door to a novel family of promising approaches complementing those relying on linear programming or iterative methods 43.
8.7.3 Optimally Solving Decentralized POMDPs Under Hierarchical Information Sharing
Participants: Sayil Barish [MS Intern], Albert David [BS Intern], Xie Yuxuan [BS Intern], Olivier Buffet [Inria Nancy, Larsen], Jilles S. Dibangoye.
This work is the result of an effort to provide a strong theory for optimally solving decentralized decision-making problems, represented as decentralized partially observable Markov decision processes (Dec-POMDPs), with tractable structures 117. This work describes core ideas of multi-agent reinforcement learning algorithms for decentralized partially observable Markov decision processes (Dec-POMDPs) under hierarchical information sharing. That is the dominant management style in our society—each agent knows what her immediate subordinate knows. In this case, we prove backups to be linear due to (normal-form) Bellman optimality equations being for the first time specified in extensive form. Doing so, one can sequence agent decisions at every point in time while still preserving optimality; hence paving the way for the first optimal -learning algorithm with linear-time updates for an important subclass of Dec-POMDPs. That is a significant improvement over the double-exponential time required for normal-form Bellman optimality equations, in which updates of agent decisions are performed in sync every point in time. Experiments show algorithms utilizing extensive instead of normal-form Bellman optimality equations can be applied effectively to much larger problems.
8.7.4 Stable Reinforcement Learning Methods
Participants: Jilles S. Dibangoye, Nicolas Marchand [GIPSA lab, Grenoble], Ahmad Hably [GIPSA lab, Grenoble], Estéban Carvalho [PhD Student, GIPSA lab, Citi lab], Vincent Andrieu [LAGEPP, Lyon], Daniele Astolfi [LAGEPP, Lyon], Giacomo Casadeic [LAGEPP, Lyon], Madiha Nadri [LAGEPP, Lyon], Samuele Zoboli [PhD Student, LAGEPP, Citi lab].
This recent research project is at the intersection of three closely related research fields: optimal control, machine learning, and mobile robotics. In this project, we try to find the best compromise between control methodologies to provide stability guarantees and machine learning methodologies that can scale up and converge towards a locally optimal solution. Applications of this hybrid methodology concern mobile robotics, including the driving of drones at very high speed. Two PhD. theses have been started on this problem the last year.
PhD thesis of Estéban Carvalho (since Oct. 2019). The first thesis focuses on high-speed navigation for UAVs in collaboration with the laboratory GIPSA. The goal of this PhD is to develop safe and aggressive piloting algorithms for UAVs. Piloting algorithms will be designed on event-switched control coupled with AI routines to cope with unmodeled and uncertain part of the system to reach as much as possible the limits of aggressive flight.
PhD thesis of Samuele Zoboli (since Jul. 2020, ANR DELICIO). In control theory, a natural approach for the design of stabilizing control is to consider a linear approximation around the operating point of the dynamic system. In this way, it is possible to design control laws that are optimal and which guaranteed stability for the linearized system. However, this behavior is only guaranteed for the original system around this operating point (local approach). The aim of this PhD thesis is to develop new real time algorithms (contol law and estimation algorithms for dynamical systems) that combines tools coming from control theory and reinforcement learning.
8.7.5 Neural Enhanced Control for Quadrotor Linear Behavior Fitting
Participants: Estéban Carvalho [PhD Student, GIPSA lab, Citi lab], Pierre Susbielle [GIPSA lab, Grenoble], Ahmad Hably [GIPSA lab, Grenoble], Jilles S. Dibangoye, Nicolas Marchand [GIPSA lab, Grenoble].
Designing an efficient autopilot for a quadrotor can be a very long and tedious process. This comes from the complex nonlinear dynamics that rule the flying robot behavior as battery discharge, blade flapping, gyroscopic effect, frictions, etc. In this paper we proposed to use a traditional cascaded control architecture enhanced with Deep Neural Network (DNN). The idea is to easily setup a control algorithm using linear cascaded laws and then correct unmodelled dynamics and approximations made during the linear control design with the DNN. The tuning process is reduced to choice of proportional and derivative gains of each control loop. The approach is tested in the ROS/Gazebo simulation environment and experimentally in a motion capture room. Results confirm that the methodology significantly improves the performance of linear approaches on nonlinear quadrotor system.
8.7.6 Neural Network Assisted Feedback Linearization Control for Quadrotor Flight with Unknown Suspended Load
Participants: Estéban Carvalho [PhD Student, GIPSA lab, Citi lab], Pierre Susbielle [GIPSA lab, Grenoble], Ahmad Hably [GIPSA lab, Grenoble], Nicolas Marchand [GIPSA lab, Grenoble], Jilles S. Dibangoye.
Load transportation problem for quadrotor is a very emerging application. The presence of a suspended load is known to generate problematic disturbances. Knowledge of the latter is often used for the design of the controller and is often based on simplifying assumptions. In order to follow a given trajectory while minimizing the perturbations of an unknown load we propose a Deep Neural Network (DNN) Assisted Feedback Linearization Controller. The DNN is used to learn load dynamics using flight data. Online estimation of these perturbation dynamics is then used to perform exact feedback linearization. The proposed approach is tested experimentally and demonstrates the effectiveness of this hybrid control architecture.
8.7.7 Locally Asymptotically Stable Deep Actor-Critic Algorithms For Quadratic Cost Setpoint Optimal Control Problems
Participants: Samuele Zoboli [PhD Student, LAGEPP, Citi lab], Vincent Andrieu [LAGEPP, Lyon], Daniele Astolfi [LAGEPP, Lyon], Giacomo Casadeic [LAGEPP, Lyon], Madiha Nadri [LAGEPP, Lyon], Jilles S. Dibangoye.
Deep reinforcement learning (DRL) algorithms showed surprising capabilities in solving complex nonlinear control problems. However, these capabilities come at a price. For instance, they construct controllers with only practical stability properties. On the contrary, Lyapunov-based controllers provide strong stability guarantees by exploiting the knowledge about the dynamic model. A recent work proved DRL algorithms, enhanced with such dynamic model information, locally guarantee desirable properties. The present work circumvents the requirement of local dynamic model's knowledge by learning it from data. The resulting DRL algorithms achieve the best of both worlds: local strong stability properties and improved domain of attraction compared to classical Lyapunov-based methods. Experiments over three benchmarks support our claims.
8.7.8 Reinforcement Learning Policies With Local LQR Guarantees For Nonlinear Discrete-Time Systems
Participants: Samuele Zoboli [PhD Student, LAGEPP, Citi lab], Vincent Andrieu [LAGEPP, Lyon], Daniele Astolfi [LAGEPP, Lyon], Giacomo Casadeic [LAGEPP, Lyon], Madiha Nadri [LAGEPP, Lyon], Jilles S. Dibangoye.
Optimal control of nonlinear systems is a difficult problem which has been addressed by both the Control Theory (CT) and Reinforcement Learning (RL) communities. Frequently, the former relies on the linearization of the system thus obtaining only local guarantees. The latter relies on data to build model-free controllers, focused solely on performances. In this paper we proposed a methodology to combine the advantages of both approaches, casting the formulation of an optimal local Linear Quadratic Regulator (LQR) into a Deep RL problem. Our solution builds on the linear framework to derive a learnt nonlinear controller showing local stability properties and global performances.
8.8 Constraint Optimisation problem
8.8.1 Constraint Programming
Participants: Jean-Yves Courtonne [Inria STEEP], François Delobel [LIMOS], Léon Fauste [LIMOS], Pascal Lafourcade [LIMOS], Luc Libralesso [Inria STEEP], Mathieu Mangeot [Inria STEEP], Loïc Rouquette, Christine Solnon.
In many of our applications, we have to solve constrained optimization problems (COPs) such as vehicle routing problems, or multi-agent path finding problems, for example. These COPs are usually NP-hard, and we study and design algorithms for solving these problems. We more particularly study approaches based on Constraint Programming (CP), that provides declarative languages for describing these problems by means of constraints, and generic algorithms for solving them by means of constraint propagation. In 23, we show how to automatically generate efficient CP models for solving differential cryptanalysis problems given the description of a symmetric block cipher.
PhD thesis of Loïc Rouquette (since Oct. 2019). This thesis is funded by the ANR project DeCrypt, that aims at designing declarative languages (based on CP) for solving cryptanalysis problems. During the first year of his thesis, Loïc has designed new propagation algorithms that are integrated in the Choco CP library. During the second year, he has designed CP models for mounting related-key differential attacks of Rijndael (paper submitted to AfricaCrypt) and Boomerang attacks of WARP (paper submitted to FSE).
PhD thesis of Léon Fauste (since Sept. 2021). This thesis is funded by Ecole Normale Supérieure Paris Saclay and it is co-supervised by Mathieu Mangeot, Jean-Yves Courtonne (from Inria STEEP team) and Christine Solnon. The goal of the thesis is to design new tools for choosing the best geographical scale when relocating productive activities, and the context of this work is described in 32. During the first months of his thesis, Léon has designed a first Integer Linear Programming (ILP) model described in 35.
8.8.2 Multi-Robot Path Planning with cable constraints
Participants: Xiao Peng [PhD. student CITI], Christine Solnon, Olivier Simonin.
The research is done within the context of the european project BugWright2 that aims at designing an adaptable autonomous multi-robot solution for servicing ship outer hulls.
PhD thesis of Xiao Peng (since Nov. 2020). The goal of this PhD thesis is to design algorithms for planning a set of mobile robots, each attached to a flexible cable, to cover an area with obstacles. In 26, we formally define the Non-Crossing MAPF (NC-MAPF) problem and show how to compute lower and upper bounds for this problem by solving well known assignment problems, we introduce a Variable Neighbourhood Search approach for improving the upper bound, and we introduce a CP model for solving the problem to optimality. In 91, we extend this approach to the case of non point-sized robots.
8.8.3 Vehicle routing problems
Participants: Romain Fontaine [PhD Student CITI], Thierry Garaix [LIMOS, Saint Etienne], Omar Rifki [LIMOS, Saint Etienne], Jilles S. Dibangoye, Christine Solnon.
The research is done within the context of the transportation challenge and funded by INSA Lyon.
Integration of traffic conditions:
In classical vehicle routing problems, travel times between locations to visit are assumed to be constant. This is not realistic because traffic conditions are not constant throughout the day, especially in an urban context. As a consequence, quickest paths (i.e., successions of road links), and travel times between locations may change along the day. To fill this lack of realism, cost functions that define travel times must become time-dependent. These time-dependent cost functions are defined by exploiting data coming from sensors, and the frequency of the measures as well as the number and the position of the sensors may have an impact on tour quality.
PhD thesis of Romain Fontaine (since Oct. 2020). State-of-the-art approaches for solving vehicle routing problems hardly scale when considering time-dependent cost functions. The goal of the PhD thesis of Romain is to design new algorithms based on dynamic programming. Constraints (such as, for example, time-windows or pickup-and-delivery constraints) are exploited to prune states, and new bounds and heuristics are designed to guide the search. The approach is described in 70.
Theoretical and empirical study of routing problems.
We study the impact of time-window constraints on the satisfiability and the difficulty of routing problems in 28.
9 Bilateral contracts and grants with industry
9.1 Bilateral contracts with industry
9.1.1 Toyota Motor Europe (2006 - 2022)
Participants: Christian Laugier, David Sierra González, Özgür Erkent, Jilles Dibangoye, Christian Wolf.The contract with Toyota Motors Europe is a joint collaboration involving Toyota Motors Europe, Inria and ProbaYes. It follows a first successful short term collaboration with Toyota in 2005. This contract aims at developing innovative technologies in the context of automotive safety. The idea is to improve road safety in driving situations by equipping vehicles with the technology to model on the fly the dynamic environment, to sense and identify potentially dangerous traffic participants or road obstacles, and to evaluate the collision risk. The sensing is performed using sensors commonly used in automotive applications such as cameras and lidar.
This collaboration has been extended in 2018 for 4 years (period 2018-2021) and Toyota provides us with an experimental vehicle Lexus equipped with various sensing and control capabilities. Several additional connected technical contracts have also been signed, and an exploitation license for the CMCDOT software has been bought by Toyota in 2018.
9.1.2 Sumitomo Electric Industries (2020-22)
Participants: Christian Laugier, Lukas Rummelhard (SED), Andres Gonzalez Moreno, Rabbia Asghar, Jerome Lussereau, Nicolas Turro (SED).R&D project (275K€) to develop and demonstrate multi-modal perception using communicating infrastructures, data exchange and fusion between such infrastructure and mobile autonomous vehicles, and the added value of such architecture. Based on embedded Bayesian perception systems (CMCDOT framework), the project aims to develop long-term prediction and risk assessment.
9.2 Bilateral grants with industry
9.2.1 Renault (2019 - 2022)
Participants: Luiz Alberto Guardini Serafim, Christian Laugier, Anne Spalanzani, Philippe Martinet.This contract was linked to the PhD Thesis of Luiz Alberto Guardini Serafim (Cifre Thesis). The objective is to address the open problem of emergency obstacle avoidance in complex traffic situations (for ADAS or AD applications) using planning in grid occupancy.
9.2.2 IRT Nanoelec – Security of Autonomous Vehicles project (2018 - 2020), Security of Industrial Robotics (2021-2023)
Participants: Christian Laugier, Lukas Rummelhard (SED), Jerome Lussereau, Andres Gonzalez Moreno, Thomas Genevois, Nicolas Turro (SED).Security of Autonomous Vehicles, then Security of Industrial Robotics, is a project supported by PIA in the scope of the program PULSE of IRT Nanoelec. The objective of this project is to integrate, develop and promote technological bricks of context capture, for the safety of the autonomous vehicle. Building on Embedded Bayesian Perception for Dynamic Environment, Bayesian data fusion and filtering technologies from sets of heterogeneous sensors, these bricks make it possible to secure the movements of vehicles, but also provide them with an enriched and useful representation for autonomy functions themselves. In this context, various demonstrators embedding those technology bricks are developed in cooperation with industrial partners.
10 Partnerships and cooperations
10.1 European initiatives
10.1.1 FP7 & H2020 projects
BugWright2 - H2020 ICT
(Autonomous Robotic Inspection and Maintenance on Ship Hulls and Storage Tanks)
- 650 K€ for Chroma and Agora teams - total 10 M€ — 2020-24
- Consortium (academic partners only): CNRS/GeorgiaTech Metz (coord), INSA-Inria Lyon, LakeSide lab. (Austria), Universität Klagenfurt, Universität Trier, Universidade de Porto, Universitat de Les Illes Balears.
- The project deals with the inspection of large infrastructures conducted by multiple heterogeneous robots (aerial, ground and underwater). Chroma coordinates the Work Package 6 on multi-robot planning, led by O. Simonin.
- link: cordis.europa.eu/project/id/871260
CPS4EU - H2020 ECSEL
(European Initiative to Enable Validation for Highly Automated Safe and Secure Systems)
- 72 K€ Europe + 101 K€ PSPC for Chroma — Period 2019-2022
- The CPS4EU consortium is composed of 36 members including large groups, medium enterprises, SMEs, Universities and RTOs from 5 countries (France, Germany, Spain, Italy, Hungary)
- CPS4EU aims to develop 4 key enabling technologies (computing, connectivity, sensing, cooperative systems), incorporate these CPS modules through pre-integrated architectures and design tools, instantiate these architectures in dedicated use cases from strategic application (automotive, smart grid and industry automation), and improve CPS awareness and usage for all industrial sectors.
10.2 National initiatives
EquipEx+ "TIRREX" (2021-30)
Participants: Olivier Simonin, Alessandro Renzaglia, Christian Laugier, Lukas Rummelhard.Title : "Technological Infrastructure for Robotics Research of Excellence" ( 12M€). TIRREX is led by CNRS and involves 32 laboratories. Chroma is involved in two research axes :
- "Aerial Robotics", Chroma/CITI lab. is a major partner of this axis (INSA Lyon & Inria)
- "Autonomous land robotics", Chroma/CITI lab. is a secondary partner of this axis (INSA Lyon & Inria).
ANR "MaestrIoT" (2021-2025)
(Multi-Agent Trust Decision Process for the Internet of Things)
Participants: Jilles Dibangoye, Olivier Simonin.Consortium: LITIS (coord.), Chroma/CITI, LCIS, IHF/LIMOS. The main objective of the MaestrIoT project is to develop an algorithmic framework for ensuring trust in a multi-agent system handling sensors and actuators of a cyber-physical environment. Trust management has to be ensured from the perception to decision making and integrating the exchange of information between WoT devices. Total funding 536 K€.
The ANR "CONCERTO" (2020-23)
Participants: Olivier Simonin, Alexandre Bonnefond.Title : "mobilité Contrôlée et cOmmunicatioNs effiCaces dans une flottE de dRones auTo-Organisée". The project is led by Isabelle Guerin-Lassous (LIP/Inria Dante). Partners : LIP (UCBL/Inria Dante), CITI (INSA/Inria Chroma), LS2N (CNRS, Nantes), LORIA (UL, Nancy), Alerion (company in Nancy). Funding : 300K€ - (ASTRID ANR).
AI Chair "REMEMBER" (2020-23) (Chair of research and teaching in artificial intelligence)
Participants: Christian Wolf (NaverLabs europe, LIRIS/INSA Lyon), Olivier Simonin, Jilles Dibangoye.Title : Learning Reasoning, Memory and Behavior (funding 574K€). This Chair is led by Christian Wolf (INSA Lyon/LIRIS lab.), who recently moved to NaverLabs Europe. It involves CITI/Chroma (O. Simonin, J. Dibangoye) and LIRIS/SyCoSMA (L. Matignon). The chair is co-financed by ANR, Naver Labs Europe and INSA-Lyon. Creating agents capable of high-level reasoning based on structured memory is the main topic of the AI Chair "REMEMBER". Funding one PhD in Chroma (P. Marza).
ANR JCJC "Plasma" (2019-2022)
Participants: Jilles Dibangoye.The ANR JCJC Plasma, led by Jilles S. Dibangoye, aims at developing a general theory and algorithms with provable guarantees to treat planning and (deep) RL problems arising from the study of multi-agent sequential decision-making, which may be described as Partially Observable Stochastic Games (POSG), see Figure 1. We shall contribute to the development of theoretical foundations of the fields of intelligent agents and MASs by characterizing the underlying structure of the multi-agent decision-making problems and designing scalable and error-bounded algorithms. The research group is made of four senior researchers, O. Simonin, C. Wolf (INSA Lyon), F. Charpillet (INRIA Nancy) and O. Buffet (INRIA Nancy), and two junior researchers Jilles S. Dibangoye and A. Saffidine (Univeristy of New South Whales). We plan to hire one PhD and one post-doc for two years as well as internships. We received a support for 42-months starting in March 2020 with a financial support of about 254K euros.
ANR "Delicio" (2019-2022)
Participants: Jilles Dibangoye, Olivier Simonin.The ANR Delicio (Data and Prior, Machine Learning and Control), led by C. Wolf (INSA Lyon, LIRIS), proposes fundamental and applied research in the areas of Machine Learning and Control with applications to drone (UAV) fleet control. The consortium is made of 3 academic partners: INSA-Lyon/LIRIS, INSA-Lyon/CITI (Chroma team), University Lyon 1/LAGEPP, and ONERA. We received a support for 48-months starting in October 2019 with a financial support of about 540K euros.
ANR "HIANIC" (2018-22)
Participants: Anne Spalanzani, Manon Predhumeau, Maria Kabtoul.The HIANIC project, led by A. Spalanzani, proposes to endow autonomous vehicles with smart behaviors (cooperation, negotiation, socially acceptable movements) that better suit complex SharedSpace situations. It will integrate models of human behaviors (pedestrian, crowds and passengers), social rules, as well as smart navigation strategies that will manage interdependent behaviors of road users and of cybercars. The consortium is made of 3 academic partners: Inria (RITS, Chroma, Pervasive Interaction teams), LIG Laboratory (Hawai team) and LS2N laboratory (ARMEN and PACCE teams).
10.2.2 Grand Defis
Grand Défi "PRISSMA" (2021-2024)
Participants: Alessandro Renzaglia, Christian Laugier, Jean-Baptiste Horel.
The project is funding 330K€ for Chroma (6.5M€ total). The PRISSMA consortium is composed of 21 French members from both industry and academia, including: UTAC (coord.), Inria (Chroma, Convecs), LNE, UGE, CEA, Navya, IRT SystemX, and others. This project aims at proposing a platform that will allow to lift the technological barriers preventing the deployment of secure AI-based systems, and to integrate all the elements necessary for the realization of the homologation activities of the autonomous vehicle and its validation in its environment for a given use case.
10.2.3 FUI Projects
FUI STAR (2018 – 2021)
Participants: Andres Gomez Hernandez, Olivier Simonin, Christian Laugier.The Project STAR is coordinated by IVECO. The academic partners of the project are Inria Grenoble-Rhône-Alpes, IFSTTAR, ISAE-Supaéro. The industrial and application partners are IVECO, Easymile, Transpolis, Transdev and Sector Groupe. The goal of the project is to build an autonomous bus that will operate on a safe lane Inria is involved in helping design situation awareness perception, especialy in special case like docking at the bus stop and handling dynamicity of any obstacle. The IRT Nanoelec is also involved in the project as a subcontractor, for testing the perception, decision-making, navigation and controls components developed in the project.
10.2.4 DGA/INRIA AI projects
Participants: Alexandre Bonnefond, Olivier Simonin, Isabelle Guerin-Lassous (Lip/Lyon 1, Inria Dante), Vincent Dufour.
The DYNAFLOCK project, led by O. Simonin, aims to extend flocking-based decentralized control of swarm of UAVs by considering the link quality between communicating entities. The consortium is made of 2 Inria teams from Lyon : Chroma and Dante (involving Prof. I. Guerin-Lassous). The PhD student (Alexandre Bonnefond) recruited in this project aims at defining dynamic flocking models based on the link quality. In 2020, an engineer will be recruited to conduct experiments with a quadrotors platform.
Funding of Dynaflock : 250 K€
10.3 Regional initiatives
WIFI-Drones project, FIL (Federation d'Informatique de Lyon), 2019-21
Participants: Isabelle Guerin-Lassous (Lip/Lyon1, Inria Dante), Olivier Simonin, Remy Grunblatt (Telecom Paris Sud).
Project between two teams of two laboratories from Lyon : DANTE (LIP) and CHROMA (CITI), entitled "Performances des communications Wi-Fi dans les réseaux de drones : une approche expérimentale", 2019-2021. Leader : I. Guerin-Lassous & O. Simonin.
The project aims to experimentally evaluate the Wireless communication in UAVs fleet scenarios. We consider the recent version of Wi-Fi based on 802.11n and 802.11 ac. Experimental measures will be used to build propagation models in order to be integrated in UAVs fleet simulations (in particular with Gazebo and NS3 simulators).
Region-funded R&D Booster "Moov-IT" project, 2020-2023
Participants: Amrita Suresh, Thomas Genevois, Christian Laugier.
The objective of the project is to develop an autonomous system to be integrated in production chains to provide modernization, flexibility and productivity, including fixed and mobile transitic systems. In that perspective, Chroma's CMCDOT framework will be adapted, extended and integrated on a mobile robot from Akeoplus, and tested in real environments.
Participants: Jilles Dibangoye, Fabrice Jumel, Christian Laugier, Agostino Martinelli, Alessandro Renzaglia, Jacques Saraydaryan, Olivier Simonin, Christine Solnon, Anne Spalanzani.
11.1 Promoting scientific activities
11.1.1 Scientific events: organisation
Member of the organizing committees
- C. Laugier co-organized in the scope of IEEE/RSJ IROS 2021, a workshop on "Perception and Navigation for Autonomous Robotics in Unstructured and Dynamic Environments" (iros2021-pnarude.github.io/index.html).
11.1.2 Scientific events: selection
Chair of conference program committees
- C. Laugier was Program Co-Chair of IEEE/RSJ IROS 2021 (Prague). He also regularly sits on the Senior Program Committee of IROS conferences.
- C. Solnon is program chair of the CP 2022 conference.
Member of the conference program committees
- J. S. Dibangoye was PC member of the NeurIPS 2021 conference.
- J. S. Dibangoye was PC member of the ICML 2021 conference.
- C. Laugier is member of the Steering Committee of the IEEE/RSJ IROS conference.
- C. Laugier is Editor (for the period 2021-2023) in the Editorial Board of the IEEE/RSJ IROS conference.
- A. Martinelli was Associate Editor for IEEE ICRA 2021.
- A. Renzaglia was Associated Editor for the IEEE IROS conference in 2021.
- A. Renzaglia was PC member of the DARS conference in 2021.
- A. Renzaglia was PC member of the ADS initiative, part of the DATE conference, in 2021.
- O. Simonin was Senior PC member of the IJCAI conference in 2021.
- O. Simonin was PC member of the ECAI conference 2021.
- O. Simonin was PC member of the AAAI conference in 2021.
- O. Simonin was PC member of the GECCO conference - Swarm Intelligence Track in 2021.
- C. Solnon was PC member of the IJCAI conference in 2021.
- C. Solnon was PC member of the AAAI conference in 2021.
- C. Solnon was PC member of the ECAI conference in 2021.
- O. Simonin, A. Martinelli, C. Laugier and A. Renzaglia serve each year as reviewer in conferences ICRA and IROS.
Member of the editorial boards
- C. Solnon is Associate Editor of Annals of Mathematics and Artificial Intelligence (AMAI).
- C. Solnon is Editorial board member of Swarm Intelligence, Neural Computing and Applications, and Constraints.
- C. Laugier is Senior Editor and member of the Steering Committee of the journal IEEE Transaction on Intelligent Vehicles.
- C. Laugier is member of the Editorial Board of the journal IEEE ROBOMECH.
- O. Simonin is a member of the editorial board of the french revue of AI since 2018, named RIA till 2020 and ROIA now.
Reviewer - reviewing activities
- O. Simonin was reviewer for TAAS (ACM Transactions on Autonomous and Adaptive Systems) and Robotics and Autonomous Systems journals.
11.1.4 Invited talks
- C. Solnon gave an invited talk at 23ème édition du congrès annuel de la Société Française de Recherche Opérationnelle et d'Aide à la Décision (ROADEF)
11.1.5 Leadership within the scientific community
- C. Laugier is regularly involved in various IEEE Awards Committees and evaluation committees: Best conference paper (ICRA, IROS, ICARCV), IEEE RAS Awards (DSA, chapter award), IROS awards (Fellow, Harashima, DSA, Young Professional), European projects applications.
- C. Laugier is member of the scientific committee of the French GDR Robotics.
- C. Solnon is member of the executive committee and the scientific committee of the AI GDR (national research group).
11.1.6 Scientific expertise
- O. Simonin was a member of the committee of the Robotics ASTRID ANR project selection in 2021.
- C. Solnon was member of the Hcéres evaluation committee of the GREYC (Caen), 2021.
- C. Solnon was expert for the Automn University of the A2U Alliance (alliance between "Université du Littoral et Cote d'Opale", "Université d’Artois" and "Université de Picardie Jules Verne")
- C. Solnon was member of the scientific committee of the "Prix de thèse du GT2L", 2021
11.1.7 Research administration
- O. Simonin is member of the Inria Lyon COS (comité d'orientation scientifique du Centre).
11.2 Teaching - Supervision - Juries
- INSA Lyon 5th year / Master - Robotics option (25 students) : AI for Robotics, Multi-Robot Systems, Robotics Projects, 90h, Resp., Telecom Dept.
- INSA Lyon 4th year / Master. : Introduction to AI, 15h (90 students), Telecom Dept.
- INSA Lyon 3rd year / Lic. : Algorithmics, 50h (100 students), L3, Resp., Telecom Dept.
Jilles S. Dibangoye :
- INSA Lyon 4th year / Master : Introduction to AI, 20h (90 students), Resp., Telecom Dept.
- INSA Lyon 3rd year / Lic. : Algorithmics, 32h, Telecom Dept.
- INSA Lyon 3rd year / Lic. & Master : Operating Systems, 56h L3 and 16h Master, Telecom Dept.
- INSA Lyon 5th year / Master : AI for Robotics 18h, Telecom Dept.
Christine Solnon :
- INSA Lyon 3rd year / Lic. : Advanced algorithms for artificial intelligence and graphs, 40h (130 students), Resp., Computer Science Dept.
- INSA Lyon 4th year / Master : Object Oriented and Agile software development, 62h (130 students), Resp., Computer Science Dept.
- INSA Lyon 5th year / Master : Fundamental Computer Science, 20h (60 students), Resp., Computer Science Dept.
- INSA Lyon 5th year / Master : Prescriptive Data Analytics, 6h (30 students), Computer Science Dept.
Fabrice Jumel :
- CPE Lyon 4-5th year / Master : Robotics option, 400h, Resp., SN Dept.
- CPE Lyon 4-5th year / Master : Robotic vision, cognitive science, HRI, deeplearning, robotic platforms, Kalman Filter, 250h., SN Dept.
Jacques Saraydaryan :
- CPE Lyon 5th year / Master : Autonomous Robot Navigation 18h, Particle Filter 12h, Robotics Middleware 16h, SN Dept.
- CPE Lyon 4-5th year / Master : Software Design and Big Data, 400h, Resp., SN Dept.
- CPE Lyon 4-5th year / Master : Software Architecture, 140h, SN Dept.
- CPE Lyon 4-5th year / Master : Introduction to Cyber Security, 48h, SN Dept.
Anne Spalanzani :
- INPG Ense3 Grenoble, Master MARS: Autonomous Robot Navigation 9h.
- Master (M2R) MoSIG: Autonomous Robotics, 12h, ENSIMAG Grenoble.
- INPG Ense3 Grenoble, Master MARS: AI and Autonomous Systems, 4h.
- INSA Lyon 5th year / Master - Robotics option: Multi-Robot Systems, 2h, Telecom Dept.
- INSA Lyon 3rd year / Lic.: supervision of research initiation projects (groups of 5 students) along one semester, 15h, Telecom Dept.
Phd in progress :
- Benoit Renault, Navigation coopérative et sociale de robots mobiles en environnement modifiable, O. Simonin and J. Saraydaryan.
- Edward Beeching, Large-scale automatic learning of autonomous agent behavior with structured deep reinforcement learning, C. Wolf, O. Simonin and J. Dibangoye.
- Luiz Serafim-Guardini, Conduite Automobile Autonome : Utilisation de grilles d'occupation probabilistes dynamiques pour la planification contextualisée de trajectoire d'urgence à criticité minimale, A. Spalanzani, C. Laugier, P. Martinet (Inria acentauri).
- Alexandre Bonnefond, Large-scale automatic learning of autonomous agent behavior with structured deep reinforcement learning, O. Simonin and I. Guerrin-Lassous (Inria Dante).
- Mihai Popescu, Planification multi-agents pour la patrouille d'un réseau de cibles découvertes dynamiquement, O. Simonin, A. Spalanzani, F. Valois (CITI/Inria Agora) (on break in 2020).
- Estéban Carvalho, Safe and aggressive piloting of UAVs, 2019, Ahmad Hably (Gipsa-Lab), Nicolas Marchand (Gipsa-Lab), Jilles S. Dibangoye.
- Xiao Peng, Planification sous contraintes d’une flotte de robots mobiles pour des tâches de couverture spatiale, O. Simonin, C. Solnon, European H2020 BugWright2 project funding.
- Pierre Marza, Apprentissage large-échelle d'agents autonomes capables de naviguer dans des environnements réels, C. Wolf, L. Matignon, O. Simonin, funded by Chaire IA "REMEMBER".
- Loic Rouquette, Programmation par contraintes pour la cryptanalyse symétrique, M. Minier (LORIA) and C. Solnon
- Romain Fontaine, Optimisation de la mobilité en ville en tenant compte des conditions de traffic, J. Dibangoye and C. Solnon
- Léon Fauste, Aide à la décision pour le choix de l'échelle lors de la relocalisation d'activités industrielles, M. Mangeot (STEEP, Inria Montbonnot), J.-Y. Courtonne (STEEP, Inria Montbonnot), and C. Solnon
PhD thesis :
- O. Simonin was reviewer and member of the defense committee of the PhD thesis of
- Bruno Dato, IRIT Univ. Toulouse 3, July 9th, 2021.
- Cristino de Souza Junior, Heudiasyc UTC, April 19th, 2021.
- Rémi Dromnelle, ISIR Sorbonne Univ., July 8th, 2021.
- Guilhem Buisan, LAAS INSA Toulouse, July 6th, 2021.
- C. Solnon was president of the defense committee of the PhD thesis of Clément Hénin, INSA Lyon, October, 2021.
- C. Solnon was external reviewer of the defense committee of the PhD thesis of
- Arthur Godet, Ecole des Mines de Nantes, September, 2021;
- Matthieu Py, Université Aix Marseille, December, 2021;
- Gustav Bjordal, Uppsala University (Sweeden), April, 2021;
- Lucas Groleaz, INSA Lyon, June, 2021.
- A. Spaalanzani was reviewer and member of the defense committee of the PhD thesis of
- Fabien Grzeskowiak, Univ. Rennes 1, June 2021.
- Guillaume Vailland, Univ. Rennes 1, Décember 2021.
11.3.1 Articles and contents
- C. Laugier participated as a speaker in two round tables broadcast on the internet, to popularize the issues and approaches on which the concept of “Autonomous Driving” is based. These round tables were organized by IRT (Institute for Technological Research) Nanoelec and the Minalogic competitiveness cluster (respectively in Sept. 2020 and Dec. 2021).
- C. Solnon has written an article on “Where are the hard problems?” for the Tangente journal 15, 2021
12 Scientific production
12.1 Major publications
- 1 articleSolving Multi-Agent Routing Problems Using Deep Attention Mechanisms.IEEE Transactions on Intelligent Transportation SystemsJuly 2021, 1-10
- 2 incollectionHandbook of Robotics 2nd edition, Chapter 62 on ''Intelligent Vehicles''.Handbook of Robotics 2nd EditionSpringer VerlagJuly 2016
- 3 inproceedingsSemantic Grid Estimation with a Hybrid Bayesian and Deep Neural Network Approach.IROS 2018 - IEEE/RSJ International Conference on Intelligent Robots and SystemsMadrid, SpainIEEEOctober 2018, 1-8
- 4 inproceedingsTowards Proactive Navigation: A Pedestrian-Vehicle Cooperation Based Behavioral Model.2020 IEEE International Conference on Robotics and Automation (ICRA)IEEE International Conference on Robotics and Automation ProceedingsParis, FranceMay 2020, 6958-6964
- 5 articleCooperative Visual-Inertial Odometry: Analysis of Singularities, Degeneracies and Minimal Cases.IEEE Robotics and Automation Letters522020, 668 - 675
- 6 articleNonlinear Unknown Input Observability: Extension of the Observability Rank Condition.IEEE Transactions on Automatic Control641January 2019, 222 - 237
- 7 inproceedingsSolving the Non-Crossing MAPF with CP.CP 2021 - 27th International Conference on Principles and Practice of Constraint ProgrammingLIPIcs: Leibniz International Proceedings in InformaticsMontpellier (on line), FranceOctober 2021, 1-17
- 8 inproceedingsAn Agent-Based Model to Predict Pedestrians Trajectories with an Autonomous Vehicle in Shared Spaces.Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2021)AAMAS 2021 - 20th International Conference on Autonomous Agents and Multiagent SystemsOnline, FranceMay 2021, 1-9
- 9 articleA Common Optimization Framework for Multi-Robot Exploration and Coverage in 3D Environments.Journal of Intelligent and Robotic Systems2020
- 10 inproceedingsReinforcement Learning Policies With Local LQR Guarantees For Nonlinear Discrete-Time Systems.2021 60th IEEE Conference on Decision and Control (CDC)CDC 2021 - 60th IEEE Conference on Decision and ControlTexas, United StatesIEEEDecember 2021
12.2 Publications of the year
International peer-reviewed conferences
National peer-reviewed Conferences
Conferences without proceedings
Doctoral dissertations and habilitation theses
Reports & preprints
Other scientific publications
12.4 Cited publications
- 47 inproceedingsVehicle Localization Based on Visual Lane Marking and Topological Map Matching.ICRA 2020 - IEEE International Conference on Robotics and AutomationParis, FranceIEEEMay 2020, 258-264
- 48 phdthesisCrossing of Road Intersections: Decision-Making Under Uncertainty for Autonomous Vehicles.Comue Université Grenoble AlpesDecember 2019
- 49 incollectionValidation Framework Applied to the Decision-Making Process for an Autonomous Vehicle Crossing Road Intersections.Validation and Verification of Automated SystemsSpringer International PublishingNovember 2020, 179-205
- 50 inproceedingsValidation of Perception and Decision-Making Systems for Autonomous Driving via Statistical Model Checking.IV 2019 - 30th IEEE Intelligent Vehicles SymposiumParis, FranceIEEEJune 2019, 252-259
- 51 inproceedingsEgoMap: Projective mapping and structured egocentric memory for Deep RL.ECML-PKDD 2020 - European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databaseshttps://arxiv.org/abs/2002.02286ghent, BelgiumSeptember 2020, 1-12
- 52 inproceedingsLearning to plan with uncertain topological maps.ECCV 2020 - 16th European Conference on Computer VisionGlasgow, United KingdomAugust 2020, 1-24
- 53 inproceedingsDeep Reinforcement Learning on a Budget: 3D Control and Reasoning Without a Supercomputer.ICPR 2020 - 25th International Conference on Pattern RecognitionMilan, ItalyDecember 2020, 1-16
- 54 bookDynamic Programming.Dover Publications, Incorporated1957
- 55 phdthesisDeep multi-agent reinforcement learning for dynamic and stochastic vehicle routing problems.Université de LyonOctober 2020
- 56 articleMulti-Robot Symmetric Formations for Gradient and Hessian Estimation with Application to Source Seeking.IEEE Transactions on Robotics353June 2019, 782-789
- 57 unpublishedOn Bellman's Optimality Principle for zs-POSGs.December 2020, https://arxiv.org/abs/2006.16395v1 - working paper or preprint
- 58 inproceedingsSur le principe d'optimalité de Bellman pour les zs-POSG.JFPDA 2020 - Journées Francophones surla Planification, la Décision et l’Apprentissagepour la conduite de systèmesAngers (virtuel), FranceJune 2020, 1-3
- 59 inproceedingsnuscenes: A multimodal dataset for autonomous driving.Proceedings of the IEEE/CVF conference on computer vision and pattern recognition2020, 11621--11631
- 60 inproceedingsTowards semantic SLAM using a monocular camera.IROS2011
- 61 inproceedingsThe cityscapes dataset for semantic urban scene understanding.Proceedings of the IEEE conference on computer vision and pattern recognition2016, 3213--3223
- 62 inproceedingsInstance Segmentation with Unsupervised Adaptation to Different Domains for Autonomous Vehicles.ICARCV 2020 - 16th International Conference on Control, Automation, Robotics and VisionShenzen, ChinaDecember 2020, 1-7
- 63 articleOptimally Solving Dec-POMDPs as Continuous-State MDPs.Journal of Artificial Intelligence Research55February 2016, 443-497
- 64 inproceedingsLearning to Act in Decentralized Partially Observable MDPs.ICML 2018 - 35th International Conference on Machine Learning80Proceedings of Machine Learning ResearchStockholm, SwedenPMLR2018, 1233-1242
- 65 articleSemantic Segmentation with Unsupervised Domain Adaptation Under Varying Weather Conditions for Autonomous Vehicles.IEEE Robotics and Automation LettersMarch 2020, 1-8
- 66 articleEnd-to-End Learning of Semantic Grid Estimation Deep Neural Network with Occupancy Grids.Unmanned systems73July 2019, 171-181
- 67 inproceedingsSemantic Grid Estimation with Occupancy Grids and Semantic Segmentation Networks.ICARCV 2018 - 15th International Conference on Control, Automation, Robotics and VisionSingapore, SingaporeNovember 2018, 1-6
- 68 inproceedingsSemantic Grid Estimation with a Hybrid Bayesian and Deep Neural Network Approach.IROS 2018 - IEEE/RSJ International Conference on Intelligent Robots and SystemsMadrid, SpainIEEEOctober 2018, 1-8
-POMDPs have Lipschitz-Continuous -Optimal Value Functions.NIPS 2018 - Thirty-second Conference on Neural Information Processing SystemsMontréal, CanadaDecember 2018
- 70 inproceedingsApproche hybride de résolution pour le Time-Dependent Traveling Salesman Problem with Time Windows.23ème congrès annuel de la Société Française de Recherche Opérationnelle et d'Aide à la DécisionINSA LyonVilleurbanne - Lyon, FranceFebruary 2022
- 71 inproceedingsContext Aware Robot Architecture, Application to the RoboCup@Home Challenge.RoboCup symposiumMontreal, CanadaJune 2018, 1-12
- 72 inproceedingsMapping likelihood of encountering humans: application to path planning in crowded environment.The European Conference on Mobile Robotics (ECMR)Proceedings of ECMR 2017Paris, FranceSeptember 2017
- 73 inproceedingsProactive Longitudinal Velocity Control In Pedestrians-Vehicle Interaction Scenarios.ITSC 2020 - 23rd IEEE International Conference on Intelligent Transportation SystemsRhodes, GreeceIEEESeptember 2020, 1-6
- 74 inproceedingsTowards Proactive Navigation: A Pedestrian-Vehicle Cooperation Based Behavioral Model.ICRA 2020 - IEEE International Conference on Robotics and AutomationIEEE International Conference on Robotics and Automation ProceedingsParis, FranceMay 2020, 6958-6964
- 75 articleSimultaneous State Initialization and Gyroscope Bias Calibration in Visual Inertial aided Navigation.IEEE Robotics and Automation LettersJanuary 2016
- 76 bookPlanning Algorithms.Available at http://planning.cs.uiuc.edu/Cambridge, U.K.Cambridge University Press2006
- 77 inproceedingsA journey in the history of Automated Driving.IROS 2019 - IEEE/RSJ International Conference on Intelligent Robots and SystemsInvited Pionneer's talk at IEEE/RSJ IROS 2019Macau, ChinaIEEENovember 2019, 1-27
- 78 inproceedingsDynamic Traffic Scene Understanding using Bayesian Sensor Fusion and Motion Prediction.ECCV 2018 - Workshop on Vision-based Navigation for Autonomous DrivingMunich, GermanySeptember 2018, 1-35
- 79 inproceedingsImpact of AI on Autonomous Driving.WRC 2019 - WRC 2019- IEEE World Robot ConferenceBeijing, ChinaIEEEAugust 2019, 1-27
- 80 inproceedingsFormal Validation of Probabilistic Collision Risk Estimation for Autonomous Driving.CIS-RAM 2019 - 9th IEEE International Conference on Cybernetics and Intelligent Systems (CIS) Robotics, Automation and Mechatronics (RAM)Bangkok, ThailandIEEENovember 2019, 1-6
- 81 articleClosed-form solution of visual-inertial structure from motion.International Journal of Computer Vision1062January 2014, 138--152
- 82 articleCooperative Visual-Inertial Odometry: Analysis of Singularities, Degeneracies and Minimal Cases.IEEE Robotics and Automation Letters522020, 668 - 675
- 83 articleNonlinear Unknown Input Observability and Unknown Input Reconstruction: The General Analytical Solution.arXiv preprint arXiv:2201.076102022
- 84 articleNonlinear Unknown Input Observability: Extension of the Observability Rank Condition.IEEE Transactions on Automatic Control, in Press2019
- 85 bookObservability: A New Theory Based on the Group of Invariance.Society for Industrial and Applied Mathematics - SIAMJune 2020, 1-259
- 86 articleCooperative Visual-Inertial Sensor Fusion: the Analytic Solution.IEEE Robotics and Automation Letters, in Press422019
- 87 articleCooperative Visual-Inertial Sensor Fusion: Fundamental Equations and State Determination in Closed-Form.Autonomous Robots2019, 1-19
- 88 articleVision and IMU Data Fusion: Closed-Form Solutions for Attitude, Speed, Absolute Scale and Bias Determination.IEEE Transactions on Robotics281July 2011, 44--60
- 89 inproceedingsMulti-Robot Simultaneous Coverage and Mapping of Complex Scene - Comparison of Different Strategies.AAMAS 2018 - 17th International Conference on Autonomous Agents and Multiagent Systems - Robotics TrackStockholm, SwedenACMJuly 2018, 559-567
- 90 articleFlocking for multi-agent dynamic systems: algorithms and theory.IEEE Transactions on Automatic Control5132006, 401-420
- 91 inproceedingsSolving the Non-Crossing MAPF for non point-sized robots.23ème congrès annuel de la Société Française de Recherche Opérationnelle et d'Aide à la DécisionINSA LyonVilleurbanne - Lyon, FranceFebruary 2022
- 92 inproceedingsProbabilistic Collision Risk Estimation for Autonomous Driving: Validation via Statistical Model Checking.IV 2020 -- 31st IEEE Intelligent Vehicles SymposiumLas Vegas, NV, United StatesOctober 2020, 1-7
- 93 inproceedingsGndNet: Fast Ground Plane Estimation and Point Cloud Segmentation for Autonomous Vehicles.IROS 2020 - IEEE/RSJ International Conference on Intelligent Robots and SystemsLas Vegas, NV, United StatesIEEEOctober 2020, 2150-2156
- 94 inproceedingsAttentional PointNet for 3D-Object Detection in Point Clouds.CVPR 2019 - Workshop on Autonomous drivingLong Beach, California, United StatesJune 2019, 1-10
- 95 inproceedingsAdapting the Social Force Model for Low Density Crowds in Open Environments.15th Social Simulation ConferenceMainz, GermanySeptember 2019
- 96 inproceedingsModeling and Simulating Pedestrian Social Group Behavior with Heterogeneous Social Relationships.SCS 2020 - Spring Simulation ConferenceVirtual event, United StatesMay 2020
- 97 inproceedingsModeling a Social Placement Cost to Extend Navigation Among Movable Obstacles (NAMO) Algorithms.IROS 2020 - IEEE/RSJ International Conference on Intelligent Robots and SystemsDOI is not yet properly functionnal, go to IEEEXplore directly : https://ieeexplore.ieee.org/abstract/document/9340892Las Vegas, United StatesOct 2020, 11345-11351
- 98 articleSearch and Localization of a Weak Source with a Multi-Robot Formation.Journal of Intelligent and Robotic Systems9732020, 623-634
- 99 articleA Common Optimization Framework for Multi-Robot Exploration and Coverage in 3D Environments.Journal of Intelligent and Robotic Systems2020
- 100 inproceedingsCombining Stochastic Optimization and Frontiers for Aerial Multi-Robot Exploration of 3D Terrains.IROS 2019 - IEEE/RSJ International Conference on Intelligent Robots and SystemsMacau, ChinaNovember 2019
- 101 inproceedingsConditional Monte Carlo Dense Occupancy Tracker.18th IEEE International Conference on Intelligent Transportation SystemsLas Palmas, SpainSeptember 2015
- 102 inproceedingsNavigation in Human Flows : Planning with Adaptive Motion Grid.IROS Workshop CrowdNavParis, FranceOctober 2018
- 103 inproceedingsPeople management framework using a 2D camera for human-robot social interactions.RoboCup 2019 - 23rd Annual RoboCup International SymposiumRobocup 2019: Robot World Cup XXIIISydney, AustraliaJuly 2019, 1-13
- 104 inproceedingsEmploying Severity of Injury to Contextualize Complex Risk Mitigation Scenarios.IV 2020 -- 31st IEEE Intelligent Vehicles SymposiumLas Vegas / Virtual, United StatesIEEEOctober 2020, 1-7
- 105 mastersthesisDomain adaptation for cross-sensor 3D object detection on point-clouds.MA ThesisUniversite Grenoble AlpesAugust 2020
- 106 inproceedingsModeling Driver Behavior From Demonstrations in Dynamic Environments Using Spatiotemporal Lattices.ICRA 2018 - Proceedings of the 2018 IEEE International Conference on Robotics and AutomationBrisbane, AustraliaMay 2018, 3384-3390
- 107 inproceedingsHuman-Like Decision-Making for Automated Driving in Highways.ITSC 2019 - 22nd IEEE International Conference on Intelligent Transportation SystemsAuckland, New ZealandOctober 2019, 1-8
- 108 inproceedingsLeveraging Dynamic Occupancy Grids for 3D Object Detection in Point Clouds.ICARCV 2020 - 16th IEEE International Conference on Control, Automation, Robotics and VisionShenzhen, ChinaIEEEDecember 2020, 1-6
- 109 phdthesisTowards Human-Like Prediction and Decision-Making for Automated Vehicles in Highway Scenarios.Université Grenoble AlpesApril 2019
- 110 inproceedingsCNN-SLAM: Real-time dense monocular SLAM with learned depth prediction.CVPR2017
- 111 articleSolving Polynomial Systems via a Stabilized Representation of Quotient Algebras.SIAM Journal on Matrix Analysis and Applications393October 2018, 1421--1447
- 112 articleOptimized flocking of autonomous drones in confined environments.Science Robotics32018, 2470-2476
- 113 inproceedingsBuilding Prior Knowledge: A Markov Based Pedestrian Prediction Model Using Urban Environmental Data.ICARCV 2018 - 15th International Conference on Control, Automation, Robotics and VisionSingapore, SingaporeNovember 2018, 1-12
- 114 bookTheory of Games and Economic Behavior.Princeton University Press1944
- 115 bookTheory of games and economic behavior.Princeton University Press2007
- 116 articleSolov2: Dynamic and fast instance segmentation.Advances in Neural information processing systems332020, 17721--17732
- 117 inproceedingsOptimally Solving Two-Agent Decentralized POMDPs Under One-Sided Information Sharing.ICML 2020 - 37th International Conference on Machine LearningVienne / Virtual, AustriaJuly 2020, 1-10
- 118 inproceedingsDriving Behavior Assessment and Anomaly Detection for Intelligent Vehicles.CIS-RAM 2019 - 9th IEEE International Conference on Cybernetics and Intelligent Systems (CIS) Robotics, Automation and Mechatronics (RAM)Bangkok, ThailandNovember 2019, 1-6