2020
Activity report
Project-Team
RITS
RNSR: 201421133P
Reseach center
Team name:
Robotics & Intelligent Transportation Systems
Domain
Perception, Cognition and Interaction
Theme
Robotics and Smart environments
Creation of the Team: 2014 February 17, updated into Project-Team: 2015 July 01

# Keywords

• A1.5. Complex systems
• A1.5.1. Systems of systems
• A1.5.2. Communicating systems
• A2.3. Embedded and cyber-physical systems
• A3.4. Machine learning and statistics
• A3.4.1. Supervised learning
• A3.4.5. Bayesian methods
• A3.4.6. Neural networks
• A3.4.8. Deep learning
• A5.3. Image processing and analysis
• A5.3.4. Registration
• A5.4. Computer vision
• A5.4.1. Object recognition
• A5.4.4. 3D and spatio-temporal reconstruction
• A5.4.5. Object tracking and motion analysis
• A5.4.6. Object localization
• A5.5.1. Geometrical modeling
• A5.9. Signal processing
• A5.10. Robotics
• A5.10.2. Perception
• A5.10.3. Planning
• A5.10.4. Robot control
• A5.10.5. Robot interaction (with the environment, humans, other robots)
• A5.10.6. Swarm robotics
• A5.10.7. Learning
• A6. Modeling, simulation and control
• A6.1. Methods in mathematical modeling
• A6.2.3. Probabilistic methods
• A6.2.6. Optimization
• A6.4.1. Deterministic control
• A6.4.3. Observability and Controlability
• A6.4.4. Stability and Stabilization
• A8.6. Information theory
• A8.9. Performance evaluation
• A9.2. Machine learning
• A9.5. Robotics
• A9.7. AI algorithmics
• B5.6. Robotic systems
• B6.6. Embedded systems
• B7.2. Smart travel
• B7.2.1. Smart vehicles
• B9.5.6. Data science

# 1 Team members, visitors, external collaborators

## Research Scientists

• Fawzi Nashashibi [Team leader, Inria, Researcher, HDR]
• Guy Fayolle [Inria, Emeritus]
• Jean-Marc Lasgouttes [Inria, Researcher]
• Gérard Le Lann [Inria, Emeritus]
• Anne Verroust-Blondet [Inria, Researcher, HDR]
• Raoul de Charette [Inria, Researcher, from Jun 2020]

## PhD Students

• Anh Quan Cao [Inria, from Nov 2020]
• Maximilian Jaritz [Valeo, until Jun 2020]
• Imane Mahtout [Renault]
• Kaouther Messaoud [Inria]
• Fabio Pizzati [Vislab]
• Renaud Poncelet [Sorbonne Université]
• Luis Guillermo Roldao Jimenez [Akka Technologies]

## Technical Staff

• Di Ai [Inria, Engineer, from Oct 2020 until Dec 2020]
• Said Alexander Alvarado Marin [Inria, Engineer, until Sep 2020]
• Sylia Baraka [Inria, Engineer, until Aug 2020]
• Farouk Ghallabi [Inria, Engineer]
• Kathia Melbouci [Inria, Engineer]
• Danut-Ovidiu Pop [Inria, Engineer, until Oct 2020]
• Armand Yvet [Inria, Engineer, until Feb 2020]
• Pierre de Beaucorps [Inria, Engineer, until Mar 2020]
• Raoul de Charette [Inria, Engineer, until May 2020]

## Interns and Apprentices

• Pranav Agarwal [Inria, until Jan 2020]
• Di Ai [Inria, from Apr 2020 until Sept 2020]
• Gabriel Faivre [Inria, from Mar 2020 until Aug 2020]
• Manuel Gonzalez [Inria, until Feb 2020]
• Leonardo Ward [Inria, until Feb 2020]

• Nathalie Gaudechoux [Inria]
• Anne Mathurin [Inria]

## Visiting Scientist

• Carlos Hidalgo [Tecnalia, until May 2020]

## External Collaborator

• Itheri Yahiaoui [Université de Reims Champagne-Ardennes]

# 2 Overall objectives

The focus of the project-team is to develop the technologies linked to Intelligent Transportation Systems (ITS) with the objective to achieve sustainable mobility by the improvement of the safety, the efficiency of road transport according to the recent “Intelligent Vehicle Initiative” launched by the DG Information Society of the European Commission (for “Smarter, Cleaner, and Safer Transport”). More specifically, we want to develop, demonstrate and test some innovative technologies covering all the advanced driver assistance systems (ADAS) and the traffic management systems going all the way to fully automated vehicles.

These developments are all based on the sciences and technologies of information and communications (STIC) and have the objective to bring significant improvements in the road transport sector through incremental or breakthrough innovations. The project-team covers fundamental R&D work on key technologies, applied research to develop techniques that solve specific problems, and demonstrator activities to evaluate and disseminate the results.

The scientific approach is focused on the analysis and optimization of road transport systems through a double approach:

1. the control of individual road vehicles to improve locally their efficiency and safety,
2. the design and control of large transportation systems.

The first theme on vehicle control is broadly based on signal processing and data fusion in order to have a better machine understanding of the situation a vehicle may encounter, and on robotics techniques to control the vehicle in order to help (or replace) the driver to avoid accidents while improving the performance of the vehicle (speed, comfort, mileage, emissions, noise...). The theme also includes software techniques needed to develop applications in a real-time distributed and complex environment with extremely high safety standards. In addition, data must be exchanged between the vehicles; communication protocols have thus to be adapted to and optimized for vehicular networks characteristics (e.g. mobility, road safety requirements, heterogeneity, density), and communication needs (e.g. network latency, quality of service, network security, network access control).

The second theme on modeling and control of large transportation systems is also largely dependent on STIC. The objective, there, is to improve significantly the performance of the transportation system in terms of throughput but also in terms of safety, emissions, energy while minimizing nuisances. The approach is to act on demand management (e.g. through information, access control or road charging) as well as on the vehicles coordination. Communications technologies are essential to implement these controls and are an essential part of the R&D, in particular in the development of technologies for highly dynamic networks.

In order to address those issues simultaneously, RITS is organized into three research axes, each of which being driven by a separate sub-team. The first axis addresses the traditional problem of vehicle guidance and autonomous navigation. The second axis focuses on the large scale deployment and the traffic analysis and modeling. The third axis deals with the problem of telecommunications from two points of view:

• Technical: design certified architectures enabling safe vehicle-to-vehicle and vehicle-to-vehicle communications obeying to standards and norm;
• Fundamental, design and develop appropriate architectures capable of handling thorny problems of routing and geonetworking in highly dynamic vehicular networks and high speed vehicles.

Of course, these three research sub-teams interact to build intelligent cooperative mobility systems.

# 3 Research program

## 3.1 Vehicle guidance and autonomous navigation

Participants: Pranav Agarwal, Said Alexander Alvarado Marin, Sylia Baraka, Pierre de Beaucorps, Anh Quan Cao, Raoul de Charette, Farouk Ghallabi, Manuel Gonzalez, Maximilian Jaritz, Imane Mahtout, Kathia Melbouci, Kaouther Messaoud, Fawzi Nashashibi, Fabio Pizzati, Renaud Poncelet, Danut Ovidiu Pop, Luis Roldao, Anne Verroust-Blondet, Leonardo Ward, Itheri Yahiaoui.

There are three basic ways to improve the safety of road vehicles and these ways are all of interest to the project-team. The first way is to assist the driver by giving him better information and warning. The second way is to take over the control of the vehicle in case of mistakes such as inattention or wrong command. The third way is to completely remove the driver from the control loop.

All three approaches rely on information processing. Only the last two involve the control of the vehicle with actions on the actuators, which are the engine power, the brakes and the steering. The research proposed by the project-team is focused on the following elements:

• perception of the environment,
• planning of the actions,
• real-time control.

## 3.2 Perception of the road environment

Participants: Anh Quan Cao, Raoul de Charette, Maximilian Jaritz, Farouk Ghallabi, Kaouther Messaoud, Fawzi Nashashibi, Fabio Pizzati, Danut Ovidiu Pop, Luis Roldao, Anne Verroust-Blondet, Itheri Yahiaoui.

Either for driver assistance or for fully automated guided vehicle purposes, the first step of any robotic system is to perceive the environment in order to assess the situation around itself. Proprioceptive sensors (accelerometer, gyrometer,...) provide information about the vehicle by itself such as its velocity or lateral acceleration. On the other hand, exteroceptive sensors, such as video camera, laser or GPS devices, provide information about the environment surrounding the vehicle or its localization. Obviously, fusion of data with various other sensors is also a focus of the research.

The following topics are already validated or under development in our team:

• relative ego-localization with respect to the infrastructure, i.e. lateral positioning on the road can be obtained by mean of vision (lane markings) and the fusion with other devices (e.g. GPS);
• global ego-localization by considering GPS measurement and proprioceptive information, even in case of GPS outage;
• road detection by using lane marking detection and navigable free space;
• detection and localization of the surrounding obstacles (vehicles, pedestrians, animals, objects on roads, etc.) and determination of their behavior can be obtained by the fusion of vision, laser or radar based data processing;
• simultaneous localization and mapping as well as mobile object tracking using laser-based and stereovision-based (SLAMMOT) algorithms.

Scene understanding is a large perception problem. In this research axis we have decided to use only computer vision as cameras have evolved very quickly and can now provide much more precise sensing of the scene, and even depth information. Two types of hardware setups were used, namely: monocular vision or stereo vision to retrieve depth information which allow extracting geometry information.

We have initiated several works:

• estimation of the ego motion using monocular scene flow. Although in the state of the art most of the algorithms use a stereo setup, researches were conducted to estimate the ego-motion using a novel approach with a strong assumption.
• bad weather conditions evaluations. Most often all computer vision algorithms work under a transparent atmosphere assumption which assumption is incorrect in the case of bad weather (rain, snow, hail, fog, etc.). In these situations the light ray are disrupted by the particles in suspension, producing light attenuation, reflection, refraction that alter the image processing.
• deep learning for object recognition. New works are being initiated in our team to develop deep learning recognition in the context of heterogeneous data.
• deep learning for vehicle motion prediction.

## 3.3 Planning and executing vehicle actions

Participants: Pierre de Beaucorps, Imane Mahtout, Fawzi Nashashibi, Renaud Poncelet, Anne Verroust-Blondet.

From the understanding of the environment, thanks to augmented perception, we have either to warn the driver to help him in the control of his vehicle, or to take control in case of a driverless vehicle. In simple situations, the planning might also be quite simple, but in the most complex situations we want to explore, the planning must involve complex algorithms dealing with the trajectories of the vehicle and its surroundings (which might involve other vehicles and/or fixed or moving obstacles). In the case of fully automated vehicles, the perception will involve some map building of the environment and obstacles, and the planning will involve partial planning with periodical recomputation to reach the long term goal. In this case, with vehicle to vehicle communications, what we want to explore is the possibility to establish a negotiation protocol in order to coordinate nearby vehicles (what humans usually do by using driving rules, common sense and/or non verbal communication). Until now, we have been focusing on the generation of geometric trajectories as a result of a maneuver selection process using grid-based rating technique or fuzzy technique. For high speed vehicles, Partial Motion Planning techniques we tested, revealed their limitations because of the computational cost. The use of quintic polynomials we designed, allowed us to elaborate trajectories with different dynamics adapted to the driver profile. These trajectories have been implemented and validated in the JointSystem demonstrator of the German Aerospace Center (DLR) used in the European project HAVEit, as well as in RITS's electrical vehicle prototype used in the French project ABV. HAVEit was also the opportunity for RITS to take in charge the implementation of the Co-Pilot system which processes perception data in order to elaborate the high level command for the actuators. These trajectories were also validated on RITS's cybercars. However, for the low speed cybercars that have pre-defined itineraries and basic maneuvers, it was necessary to develop a more adapted planning and control system. Therefore, we have developed a nonlinear adaptive control for automated overtaking maneuver using quadratic polynomials and Lyapunov function candidate and taking into account the vehicles kinematics. For the global mobility systems we are developing, the control of the vehicles includes also advanced platooning, automated parking, automated docking, etc. For each functionality a dedicated control algorithm was designed (see publication of previous years).

In the past few years we initiated and conducted focused research on the so-called "Plug & Play Control"' as well as on their application to the control of autonomous vehicles. Two PhD thesis (Francisco Navas Matos and Imane Mahtout) have centered their studies on the design of control architectures that are able to deal with changes in the operating conditions and assure vehicle performance and stability. Youla-Kucera parametrization has been used to design control structures able to recognize the driving situation changes, adapting the controller response to satisfy the required performance level and keeping the motion stability with a natural vehicle behavior. These architectures were validated through real implementations on RITS and Renault autonomous prototype vehicles; more specifically to develop longitudinal and lateral control of the vehicle in the presence of disturbances and context sudden modifications.

## 3.4 Cyberphysical constructs and mobile communications for fully automated networked vehicles

Participants: Gérard Le Lann.

Safety, privacy, efficiency, and cybersecurity (SPEC) properties are key to the advent of self-forming and self-healing networks of fully automated driverless terrestrial vehicles.

• Safety (S): Ratios of fatalities and non-lethal accidents should be much smaller than those achieved with non-fully automated systems that rely on human supervision or interventions.
• Privacy (P): No personal data should be inferred or extracted from cyber information, wireless communications, or from physical information such as paths and routes followed by vehicles.
• Efficiency (E): Road capacity and vehicular densities should be much higher than those achieved with non-fully automated systems for identical velocities. For example, inter-vehicular gaps are much smaller, which is antagonistic with the safety property.
• Cybersecurity (C): Safety should not be compromised by internal or external cyberattacks such as spoofing, Sybil attacks, Man-in-the-Middle attacks, message falsification or suppression, injection of bogus data, replay attacks, Denial-of-Service.

Vehicles under study are referred to as Next-Gen Vehicles (NGVs), in order to avoid confusion with Connected Autonomous Vehicles (CAVs).

CAVs match SAE levels 3 and 4. Human factors are serious impediments to achieving desired safety properties with SAE L3 vehicles. This is due to large latencies involved with human reactivity, ranging between 2 and 7 seconds, as shown in numerous experiments. SAE L4 vehicles are radically different from SAE L5 vehicles, since L4 vehicles are bound to circulate in specific lanes and neighborhoods, at rather low velocities.

NGVs are SAE L5 vehicles: No human driving or supervision, any time, everywhere. Quoting Anthony Levandowski, formerly in charge of Google cars, before Waymo was created, we believe that ”The human is the bug”. Furthermore, solving L5 problems provides leads to solutions for every other SAE level, whereas solving L3 or L4 problems does not help much in inferring solutions for L5 driving.

CAVs and NGVs rest on robotics capabilities (radars, lidars, cameras, motion control laws, learning processes, GNSS devices, actuators, etc.). CAVs are equipped with V2X (vehicle-to-everything) functionalities based on medium range WiFi DSRC communications, and cellular C-V2X communications. They rely on periodic beaconing of messages that carry their GNSS coordinates, and on V2I (vehicle-to-infrastructure) communications for accessing remote cloud-based services via road-side units or static radio relays.

NGVs will be equipped with CMX (coordinated mobility for X) functionalities, X standing for S, P, E, and C, based on very short-range directional C-V2V communications (cellular radio and optics between vehicles proximate to each other), medium range WiFi communications, and explicit agreement algorithms aimed at handling safety-critical maneuvers. NGVs do not rely at all on V2I communications relayed via terrestrial infrastructures and nodes. The CMX framework matches mobile edge computing models, whereby everything that is needed for SPEC properties is available from vehicles onboard systems.

Examples of innovative concepts at the core of the CMX framework are vehicle cyberphysical levels (which complement SAE levels), unfalsifiable vehicle profiles, and proactive security modules (extension of hardware security modules). Periodic beaconing is banished. In addition to being useless regarding safety and efficiency properties, it enables tracking and illegitimate cyber-espionage.

## 3.5 Probabilistic modeling for large transportation systems

Participants: Guy Fayolle, Jean-Marc Lasgouttes.

This activity concerns the modeling of random systems related to ITS, through the identification and development of solutions based on probabilistic methods and more specifically through the exploration of links between large random systems and statistical physics. Traffic modeling is a very fertile area of application for this approach, both for macroscopic (fleet management  39, traffic prediction) and for microscopic (movement of each vehicle, formation of traffic jams) analysis. When the size or volume of structures grows (leading to the so-called “thermodynamic limit”), we study the quantitative and qualitative (performance, speed, stability, phase transitions, complexity, etc.) features of the system.

In the recent years, several directions have been explored.

### 3.5.1 Traffic reconstruction

Large random systems are a natural part of macroscopic studies of traffic, where several models from statistical physics can be fruitfully employed. One example is fleet management, where one main issue is to find optimal ways of reallocating unused vehicles: it has been shown that Coulombian potentials might be an efficient tool to drive the flow of vehicles. Another case deals with the prediction of traffic conditions, when the data comes from probe vehicles instead of static sensors.

While the widely-used macroscopic traffic flow models are well adapted to highway traffic, where the distance between junction is long (see for example the work done by the NeCS team in Grenoble), our focus is on a more urban situation, where the graphs are much denser. The approach we are advocating here is model-less, and based on statistical inference rather than fundamental diagrams of road segments. Using the Ising model or even a Gaussian Random Markov Field, together with the very popular Belief Propagation (BP) algorithm, we have been able to show how real-time data can be used for traffic prediction and reconstruction (in the space-time domain).

This new use of BP algorithm raises some theoretical questions about the ways the make the belief propagation algorithm more efficient:

• find the best way to inject real-valued data in an Ising model with binary variables  43;
• build macroscopic variables that measure the overall state of the underlying graph, in order to improve the local propagation of information  40;
• make the underlying model as sparse as possible, in order to improve BP convergence and quality  42.

### 3.5.2 Exclusion processes for road traffic modeling

The focus here is on road traffic modeled as a granular flow, in order to analyze the features that can be explained by its random nature. This approach is complementary to macroscopic models of traffic flow (as done for example in the Opale team at Inria), which rely mainly on ODEs and PDEs to describe the traffic as a fluid.

One particular feature of road traffic that is of interest to us is the spontaneous formation of traffic jams. It is known that systems as simple as the Nagel-Schreckenberg model are able to describe traffic jams as an emergent phenomenon due to interaction between vehicles. However, even this simple model cannot be explicitly analyzed and therefore one has to resort to simulation.

One of the simplest solvable (but non trivial) probabilistic models for road traffic is the exclusion process. It lends itself to a number of extensions allowing to tackle some particular features of traffic flows: variable speed of particles, synchronized move of consecutive particles (platooning), use of geometries more complex than plain 1D (cross roads or even fully connected networks), formation and stability of vehicle clusters (vehicles that are close enough to establish an ad-hoc communication system), two-lane roads with overtaking.

The aspect that we have particularly studied is the possibility to let the speed of vehicle evolve with time. To this end, we consider models equivalent to a series of queues where the pair (service rate, number of customers) forms a random walk in the quarter plane ${ℤ}_{+}^{2}$.

Having in mind a global project concerning the analysis of complex systems, we also focus on the interplay between discrete and continuous description: in some cases, this recurrent question can be addressed quite rigorously via probabilistic methods.

We have considered in  37 some classes of models dealing with the dynamics of discrete curves subjected to stochastic deformations. It turns out that the problems of interest can be set in terms of interacting exclusion processes, the ultimate goal being to derive hydrodynamic limits after proper scaling. A seemingly new method is proposed, which relies on the analysis of specific partial differential operators, involving variational calculus and functional integration. Starting from a detailed analysis of the Asymmetric Simple Exclusion Process (ASEP) system on the torus $ℤ/nℤ$, the arguments a priori work in higher dimensions (ABC, multi-type exclusion processes, etc), leading to systems of coupled partial differential equations of Burgers' type.

### 3.5.3 Random walks in the quarter plane

This field remains one of the important violon d'Ingres in our research activities in stochastic processes, both from theoretical and applied points of view. In particular, it is a building block for models of many communication and transportation systems.

One essential question concerns the computation of stationary measures (when they exist). As for the answer, it has been given by original methods formerly developed in the team (see books and related bibliography). For instance, in the case of small steps (jumps of size one in the interior of ${ℤ}_{+}^{2}$), the invariant measure $\left\{{\pi }_{i,j},i,j\ge 0\right\}$ does satisfy the fundamental functional equation (see  2):

$Q\left(x,y\right)\pi \left(x,y\right)=q\left(x,y\right)\pi \left(x\right)+\stackrel{˜}{q}\left(x,y\right)\stackrel{˜}{\pi }\left(y\right)+{\pi }_{0}\left(x,y\right).$ 1

where the unknown generating functions $\pi \left(x,y\right),\pi \left(x\right),\stackrel{˜}{\pi }\left(y\right),{\pi }_{0}\left(x,y\right)$ are sought to be analytic in the region $\left\{\left(x,y\right)\in {ℂ}^{2}:|x|<1,|y|<1\right\}$, and continuous on their respective boundaries.

The given function $Q\left(x,y\right)={\sum }_{i,j}{p}_{i,j}{x}^{i}{y}^{j}-1$, where the sum runs over the possible jumps of the walk inside ${ℤ}_{+}^{2}$, is often referred to as the kernel. Then it has been shown that equation 1 can be solved by reduction to a boundary-value problem of Riemann-Hilbert type. This method has been the source of numerous and fruitful developments. Some recent and ongoing works have been dealing with the following matters.

• Group of the random walk. In several studies, it has been noticed that the so-called group of the walk governs the behavior of a number of quantities, in particular through its order, which is always even. In the case of small jumps, the algebraic curve $R$ defined by $\left\{Q\left(x,y\right)=0\right\}$ is either of genus 0 (the sphere) or 1 (the torus). In [Fayolle-2011a], when the drift of the random walk is equal to 0 (and then so is the genus), an effective criterion gives the order of the group. More generally, it is also proved that whenever the genus is 0, this order is infinite, except precisely for the zero drift case, where finiteness is quite possible. When the genus is 1, the situation is more difficult. Recently  38, a criterion has been found in terms of a determinant of order 3 or 4, depending on the arity of the group.
• Nature of the counting generating functions. Enumeration of planar lattice walks is a classical topic in combinatorics. For a given set of allowed jumps (or steps), it is a matter of counting the number of paths starting from some point and ending at some arbitrary point in a given time, and possibly restricted to some regions of the plane. A first basic and natural question arises: how many such paths exist? A second question concerns the nature of the associated counting generating functions (CGF): are they rational, algebraic, holonomic (or D-finite, i.e. solution of a linear differential equation with polynomial coefficients)?

Let $f\left(i,j,k\right)$ denote the number of paths in ${ℤ}_{+}^{2}$ starting from $\left(0,0\right)$ and ending at $\left(i,j\right)$ at time $k$. Then the corresponding CGF

$F\left(x,y,z\right)=\sum _{i,j,k\ge 0}f\left(i,j,k\right){x}^{i}{y}^{j}{z}^{k}$ 2

satisfies the functional equation

$K\left(x,y\right)F\left(x,y,z\right)=c\left(x\right)F\left(x,0,z\right)+\stackrel{˜}{c}\left(y\right)F\left(0,y,z\right)+{c}_{0}\left(x,y\right),$ 3

where $z$ is considered as a time-parameter. Clearly, equations 2 and 1 are of the same nature, and answers to the above questions have been given in [Fayolle-2010].

• Some exact asymptotics in the counting of walks in ${ℤ}_{+}^{2}$. A new and uniform approach has been proposed about the following problem: What is the asymptotic behavior, as their length goes to infinity, of the number of walks ending at some given point or domain (for instance one axis)? The method in [Fayolle-2012] works for both finite or infinite groups, and for walks not necessarily restricted to excursions.

### 3.5.4 Simulation for urban mobility

We have worked on various simulation tools to study and evaluate the performance of different transportation modes covering an entire urban area.

• Discrete event simulation for collective taxis, a public transportation system with a service quality comparable with that of conventional taxis.
• Discrete event simulation a system of self-service cars that can reconfigure themselves into shuttles, therefore creating a multimodal public transportation system; this second simulator is intended to become a generic tool for multimodal transportation.
• Joint microscopic simulation of mobility and communication, necessary for investigation of cooperative platoons performance.

These two programs use a technique allowing to run simulations in batch mode and analyze the dynamics of the system afterward.

# 4 Application domains

## 4.1 Introduction

While the preceding section focused on methodology, in connection with automated guided vehicles, it should be stressed that the evolution of the problems which we deal with remains often guided by the technological developments. We enumerate three fields of application whose relative importance varies with time and which have strong mutual dependencies: driving assistance, cars available in self-service mode and fully automated vehicles (cybercars).

## 4.2 Driving assistance

Several techniques will soon help drivers. One of the first immediate goal is to improve security by alerting the driver when some potentially dangerous or dangerous situations arise, i.e. collision warning systems or lane tracking could help a bus driver and surrounding vehicle drivers to more efficiently operate their vehicles. Human factors issues could be addressed to control the driver workload based on additional information processing requirements. Another issue is to optimize individual journeys. This means developing software for calculating optimal (for the user or for the community) paths. Nowadays, path planning software is based on a static view of the traffic: efforts have to be done to take the dynamic component in account.

## 4.3 New transportation systems

The problems related to the abusive use of the individual car in large cities led the populations and the political leaders to support the development of public transport. A demand exists for a transport of people and goods which associates quality of service, environmental protection and access to the greatest number. Thus the tram and the light subways of VAL type recently introduced into several cities in France conquered the populations, in spite of high financial costs. However, these means of mass transportation are only possible on lines on which there is a keen demand. As soon as one moves away from these “lines of desire” or when one deviates from the rush hours, these modes become expensive and offer can thus only be limited in space and time. To give a more flexible offer, it is necessary to plan more individual modes which approach the car as we know it. However, if one wants to enjoy the benefits of the individual car without suffering from their disadvantages, it is necessary to try to match several criteria: availability anywhere and anytime to all, lower air and soils pollution as well as sound levels, reduced ground space occupation, security, low cost. Electric or gas vehicles available in self-service, as in the Praxitèle system, bring a first response to these criteria. To be able to still better meet the needs, it is however necessary to re-examine the design of the vehicles on the following points:

• ease empty car moves to better distribute them;
• better use of information systems inboard and on ground;
• better integrate this system in the global transportation system.

These systems are now operating. The challenge is to bring them to an industrial phase by transferring technologies to these still experimental projects.

## 4.4 Automated vehicles

The long term effort of the project is to put automatically guided vehicles (cybercars) on the road. It seems too early to mix cybercars and traditional vehicles, but data processing and automation now make it possible to consider in the relatively short term the development of such vehicles and the adapted infrastructures. RITS aims at using these technologies on experimental platforms (vehicles and infrastructures) to accelerate the technology transfer and to innovate in this field. Other application can be precision docking systems that will allow buses to be automatically maneuvered into a loading zone or maintenance area, allowing easier access for passengers, or more efficient maintenance operations. Transit operating costs will also be reduced through decreased maintenance costs and less damage to the braking and steering systems. Regarding technical topics, several aspects of Cybercars have been developed at RITS this year. First, we have stabilized a generic Cycab architecture involving Inria SynDEx tool and CAN communications. The critical part of the vehicle is using a real-time SynDEx application controlling the actuators via two Motorola’s MPC555. Today, we have decided to migrate to the new dsPIC architecture for more efficiency and ease of use. This application has a second feature, it can receive commands from an external source (Asynchronously to this time) on a second CAN bus. This external source can be a PC or a dedicated CPU, we call it high level. To work on the high level, in the past years we have been developing a R&D framework called (Taxi) which used to take control of the vehicle (Cycab and Yamaha) and process data such as gyro, GPS, cameras, wireless communications and so on. In order to rely on a professional and maintained solution, we use the RTMaps SDK development platform for our developments and demonstrations. These demonstrations include: reliable SLAMMOT algorithm using 2 to 4 laser sensors simultaneously, automatic line/road following techniques, PDA remote control, multi sensors data fusion, collaborative perception via ad-hoc network. The second main topic is inter-vehicle communications using ad-hoc networks. We have worked with the EVA team for setting and tuning OLSR, a dynamic routing protocol for vehicles communications. Our goal is to develop a vehicle dedicated communication software suite, running on a specialized hardware. It can be linked also with the Taxi Framework for getting data such GPS information’s to help the routing algorithm.

# 5 Highlights of the year

Raoul de Charette obtained an ANR JCJC - Project SIGHT (viSIon throuGH weaTher) Young researcher grant of computer vision algorithms for vision in adverse weather conditions. SIGHT addresses the problem of un-/self- supervision training to improve performance of vision algorithms through rain, snow, fog handing self-supervised model-based learning for domain adaptation and continuous image-to-image translation.

# 6 New software and platforms

## 6.1 New software

### 6.1.1 PML-SLAM

• Keyword: Localization
• Scientific Description: Simultaneous Localization and Mapping method based on 2D laser data.
• Functional Description: Simultaneous Localization and Mapping method based on 2D laser data.
• Authors: Fawzi Nashashibi, Zayed Alsayed
• Contact: Fawzi Nashashibi
• Participants: Fawzi Nashashibi, Zayed Alsayed

### 6.1.2 V2Provue

• Name: Vehicle-to-Pedestrian
• Functional Description:

It is a software developed for the Vehicle-to-Pedestrian (V2P) communications, risk calculation, and alarming pedestrians of collision risk. This software is made of an Android application dedicated to pedestrians and RtMaps modules for the vehicles.

On the pedestrian side, the application is relying on GPS data to localize the user and Wi-Fi communications are used to receive messages about close vehicles and send information about the pedestrian positioning. Besides, a service has been developed to evaluate the collision risk with the vehicles near the pedestrian and an HMI based on OpenStreetMap displays all the useful information such as pedestrian and vehicles localization and, collision risk.

On the vehicle side, RtMaps modules allowing V2X communications have been developed. These modules contain features such as TCP/UDP socket transmissions, broadcast, multicast, unicast communications, routing, forwarding algorithms, and application specific modules. In the V2ProVu software, a particular application module has been implemented to create data packets containing information about the vehicle state (position, speed, yaw rate,...) and the V2X communication stack is used to broadcast these packets towards pedestrians. Moreover, the V2proVu application can also receive data from pedestrians and create objects structures that can be shared with the vehicle perception tools.

• Contact: Fawzi Nashashibi

### 6.1.3 SimConVA

• Name: Connected Autonomous Vehicles Simulator
• Functional Description:

The software provides an interface between the network simulator ns-3 (https://www.nsnam.org/) and the modular prototyping framework RTMaps (https://intempora.com/).

This code allows to create an RTMaps component which activates and controls the ns-3 simulator. The component handles the sending and reception of data packets between ns-3 and RTMaps for each vehicle. It also handles the mobility of vehicles in ns-3 using their known position in RTMaps.

• Authors: Pierre Merdrignac, Oyunchimeg Shagdar, Jean-Marc Lasgouttes
• Contacts: Jean-Marc Lasgouttes, Fawzi Nashashibi

# 7 New results

## 7.1 Physics/GAN rendering for improving robustness to bad weather

Participants: Raoul de Charette, Shirsendu Halder, Jean-Francois Lalonde, Fabio Pizzati, Maxime Tremblay.

Rain and fog fill the atmosphere with water particles, which breaks the common assumption that light travels unaltered from the scene to the camera. While it is well-known that rain affects computer vision algorithms, improving performance in adverse weathers is difficult because there are very few existing datasets including adverse conditions.

In this axis, we conducted a thorough study on the effect of fog and rain, using a rendering pipeline that enables the systematic evaluation of common computer vision algorithms to controlled amounts of rain/fog. Our work has been published in journal IJCV 2020 19, as an extension of our ICCV'19 work 4.

We developed three different ways to add synthetic rain to existing images datasets: completely physic-based; completely data-driven; and a combination of both. The physic-based rain augmentation combines a physical particle simulator and accurate rain photometric modeling. The benefit of either methods is evaluated quantitatively, qualitatively, and we also compare against real rain images using performance metrics and by conducting a user study. The latter validated our rendering methods, demonstrating our rain is judged as much as $73%$ more realistic than the state-of-the-art. Using our generated rain-augmented KITTI, Cityscapes, and nuScenes datasets, we conduct a thorough evaluation of object detection, semantic segmentation, and depth estimation algorithms and show that their performance decreases in degraded weather, on the order of $15%$ for object detection, $60%$ for semantic segmentation, and 6-fold increase in depth estimation error. Finetuning on our augmented synthetic data results in improvements of $21%$ on object detection, $37%$ on semantic segmentation, and $8%$ on depth estimation.

The code of both works was published open source (https://github.com/cv-rits/rain-rendering) and we released a dataset with close to 400k augmented images (https://team.inria.fr/rits/computer-vision/weather-augment/). Altogether code and datasets were downloaded approx. 100+ times.

The research was conducted with our scientific partner Université Laval (Québec, Canada) in the context of the joint grant obtained by R. de Charette and J-F. Lalonde.

## 7.2 Semi-/Un- supervised Domain Adaptation

Participants: Raoul de Charette, Maximilian Jaritz, Fabio Pizzati.

There is an evident dead end to the paradigm of supervised learning, as it requires costly human labeling of millions of data frames to learn the appearance models of objects. There is still an important lack of datasets encompassing adverse lighting and weather conditions and getting robust to such conditions require transferring the features learned on a source labeled dataset to a target dataset, which is coined as Domain Adaptation (DA). In Unsupervised DA (UDA) we investigate the research paradigm where the target dataset has no labels since labeling data comes at a staggering amount of time ( 1.5hr per image). In Semi-supervised DA (SSD) we investigate the more applicative case where the target dataset contains a small subset (generally less than $10%$) of labeled data and how we can benefit from the latter to infer better feature adaptation.

In this axis, we proposed 2 new methods on UDA. In the first one published in CVPR'20 22 with PhD student M. Jaritz, we study the benefit of learning from cross-modalities, and propose a cross-modal learning called xMUDA. Our proposal disentangles two heads in a two-branches 2D/3D deep architecture where either expert branch tries to predict the other modality prediction in addition to perform as best as possible with its own modality. This work outperforms all methods on multiple UDA scenarios, With PhD student F. Pizzati, we published in WACV'20 25 the ability to produce efficient pseudo-labels from a single image in an unsupervised manner to transfer learning to the target dataset. The latter includes work on image-to-image translation and a new region growing strategy for pseudo-label thresholding. The code for xMUDA 22 has been released open source (https://github.com/valeoai/xmuda).

We extended xMUDA to new UDA scenarios and to Semi-Supervised DA (SSDA). In the new work 34, we extended our work to scenarios where a few targets samples are labeled, typically like the Waymo Open Dataset. We combined here the benefit of our cross-modal loss with supervised semantic loss to boost the result. Thorough ablation studies show the benefit of our proposal and demonstrate better performance than the literature on almost all tested scenarios.

## 7.3 Image-to-image translation

Participants: Raoul de Charette, Fabio Pizzati.

Image-to-image (i2i) translation enables the transformation of an image from a source domain (e.g. clear weather) to a target domain (e.g. rainy weather), thus producing a translation of the original image. To train such i2i networks, one need paired images or unpaired set of images assuming the use of consistency losses. In this axis we initiated two new fields of research: disentangled representation for i2i, and continuous i2i.

In the first, we investigate i2i where the target domain encompasses two sub domains and propose a disentanglement model-driven mechanism. This is typically the case when the target domain (e.g. rainy image) include occlusion on the lenses or in the scene (e.g. water drops on the windshield, fog, etc.). Our unsupervised model-based learning disentangles scene and occlusions, while benefiting from an adversarial pipeline to regress physical parameters of the occlusion model. The experiments demonstrate our method is able to handle varying types of occlusions and generate highly realistic translations, qualitatively and quantitatively outperforming the state-of-the-art on multiple datasets. The latter was published in ECCV'20 24. A follow up work has been initiated encompassing new types of disentanglement, using either supervised disentanglement, or non-differentiable models with genetic-driven deep regression. It is planned to be soon submitted to a journal.

We also investigate the case of continuous i2i, where instead of translating from a source to a target domain, we wish to produce continuous translation assuming unknown and unordered target data. For this research, we applied our work to day-to-time translation and shown that our model-guidance enables continuous translations discovering – in an unsupervised manner – a efficient and consistent walk along the manifold. Thus discovering for example dawn and dusk appearance from unlabeled target images. The benefit of our method has been assessed with GAN metrics, which demonstrate better performance on all scenarios. The work has been submitted to a conference and will be published soon.

## 7.4 Sparse deep reinforcement learning for end-to-end driving

Participants: Pierre de Beaucorps, Raoul de Charette, Pranav Agarwal.

A different paradigm for autonomous driving consists into using end-to-end driving for directly learning the vehicle control from input sensors like a camera image sequences. This is typically done with reinforcement learning which enables learning a deep control policy from hours of driving in a virtual environment. Usually, such technique uses simple dense reward functions, like the distance to the lane center or the angle of the car, to reward the agent as it learns to drive. Different from this we investigate how to learn end-to-end driving using only sparse reward.

In this axis, rather than rewarding the agent at each step, we only reward it at the end of the driving scenario as it reached its goal. While this is significantly harder since the supervision signal is very weak, this enables the agent to discover by itself new driving strategies. In our work, we studied the benefit of sparse reward in conjunction of curriculum learning, which allows the agent to address progressively harder driving tasks (e.g. driving first 5m, then 10m, etc.). Finally, we learn multiple policies (drive straight, left, right) switched at inference time. The work will be published soon.

## 7.5 Fast tracking with RGB-D-E

Participants: Raoul de Charette, Etienne Dubeau, Mathieu Garon, Jean-François Lalonde.

Augmented reality devices require multiple sensors to perform various tasks such as localization and tracking. Currently, popular cameras are mostly frame-based (e.g. RGB and Depth) which impose a high data bandwidth and power usage. With the necessity for low power and more responsive augmented reality systems, using solely frame-based sensors imposes limits to the various algorithms that needs high frequency data from the environement. As such, event-based sensors have become increasingly popular due to their low power, bandwidth and latency, as well as their very high frequency data acquisition capabilities.

In this axis, we proposed in 20, for the first time, to use an event-based camera to increase the speed of 3D object tracking in 6 degrees of freedom. This application requires handling very high object speed to convey compelling AR experiences. To this end, we propose a new system which combines a recent RGB-D sensor (Kinect Azure) with an event camera (DAVIS346). We develop a deep learning approach, which combines an existing RGB-D network along with a novel event-based network in a cascade fashion, and demonstrate that our approach significantly improves the robustness of a state-of-the-art frame-based 6-DOF object tracker using our RGB-D-E pipeline.

The research was conducted with our scientific partner Université Laval (Québec, Canada) in the context of the joint grant obtained by R. de Charette and J-F. Lalonde.

## 7.6 Cooperative perception for assistant system and automated driving

Participants: Pierre de Beaucorps, Raoul de Charette, Manuel Gonzales.

In the context of a FUI Project, PACV2x 2016-2020, we have investigated cooperative perception. Rather than considering each traffic participant individually, we consider the case where participants share their own perception outputs to all cooperative vehicles via a Road Side Unit (RSU). As such, we proposed and developed algorithms to provide cooperative guidance to all near by vehicles so as to smooth the driving, and increase the traffic throughput. Because cooperation is typically useful at intersections, we investigated highway and lane merging using clustering algorithms and predictive systems for optimal trajectory proposals.

The outcome of our work was successfully tested in closed-circuit conditions (both at Inria Rocquencourt and at Satory test tracks) and was experimented in real highway conditions thanks to our industrial partners collaboration. Demo were done in the context of the project deliverables and a full-day workshop took place in Paris to share the results with officials and the research/industrial community.

## 7.7 3D Semantic Scene Completion

Participants: Luis Roldão, Raoul de Charette, Anne Verroust-Blondet, Anh Quan Cao.

A comprehensive 3D sensing of the scene is crucial for applications like mobile robotics, and more especially for autonomous driving. Recent research on Semantic Scene Completion (SSC) enables to jointly complete and semantically label entire 3D scenes, including missing and occluded regions from raw input scans. In this work, we introduce a new approach for multiscale 3D SSC from voxelized sparse 3D LiDAR scans for autonomous driving applications. As opposed to the literature, we use a 2D UNet backbone with comprehensive multiscale skip connections to enhance feature flow and decrease computation overhead, along with 3D segmentation heads to retrieve spatial information. Our method achieves performance on par with other state of the art (SoA) methods while being significantly lighter and faster.

3D data causes representation challenges for learning-based algorithms, where data is commonly encoded using 3D voxel grids processed by 3D CNNs. While this achieves good results, it requires heavy computation and memory needs. Consequently, most of the literature limits output resolution, limiting the deployment of such methods to real-time applications. We tackle this by proposing a lightweight CNN that process a 3D voxel grid with considerably lighter 2D convolutions. This is achieved by convolving along one spatial axis while mapping to the third dimension with the use of 3D segmentation heads. In our proposal, multiscale completion is also possible given informative features map flow, preserving computation efficiency and enabling very fast inference at coarse levels.

The proposed method was evaluated in challenging SSC benchmark of SemanticKITTI. Results show that our method performs on par with SoA for semantic completion (-0.69 mIoU) and significantly improves occupancy completion (+4.72 IoU), while containing significantly fewer parameters (0.35M compared to 0.93M of second lightest network). Furthermore, our method provides significantly higher inference speeds, especially at coarse resolutions thanks to its multiscale capability, achieving inference times up to 372FPS at lower scale. This research opens the door to new real-time oriented SSC methods for automated driving.

More details can be found in 27. This research is partially funded by AKKA Technology.

A new PhD student, Anh Quan Cao, has also investigated new ways to tackle the problem of semantic scene completion, using fold-based strategies for point-wise scene completion. New research directions are still investigated.

## 7.8 Attention-based vehicle trajectory prediction

Participants: Kaouther Messaoud, Fawzi Nashashibi, Anne Verroust-Blondet, Itheri Yahiaoui.

Scene understanding and future motion prediction of surrounding vehicles are crucial to achieve safe and reliable decision-making and motion planning for autonomous driving in a highway environment. This is a challenging task considering the correlation between the drivers behaviors. This year, two methods using attention mechanisms have been introduced in this context:

• In 15 an extension of our previous work 8 for highways context has been done.
• In collaboration with Laboratory for Intelligent & Safe Automobiles (LISA) team of the university of California San Diego, we tackled the task of trajectory prediction of a target vehicle in an urban environment 35. For this purpose, we developed a deep learning-based solution exploiting information about the surrounding agents and the scene context. Our solution models the temporal evolution of the target agent using a recurrent neural network. We also model the influence of the dynamic agents and the static scene context (road geometry, lane structure…) using a muti-head attention network. In order to account for the uncertainty of the future, we generate multiple predicted plausible trajectories and the probability of occurrence of each one. We also ensure that the predicted trajectory is compliant with the scene structure which means that the predicted point lie in the drivable area using an auxiliary loss function. The developed solution is trained and evaluated using the nuScenes naturalistic driving dataset. We also made a submission for the nuScenes challenge and we earned the second price. Part of this work has been done during the stay of Kaouther Messaoud in USA.

## 7.9 LPG-SLAM: a Light-weight Probabilistic Graph-based SLAM

Participants: Kathia Melbouci, Fawzi Nashashibi.

In our previous work 23, we have introduced LPG-SLAM, a light-weight probabilistic graph-based SLAM which exploits data from high-frequency 3D sensors such as Velodyne LIDAR. We showed that 2D occupancy grids obtained from projecting 3D point clouds still provide strong geometric constraints that are sufficient for accurate localization on challenging benchmarks. Unlike most state-of-the-art methods which require powerful hardware such as GPUs, our formulation results in a lower-dimensional optimization problem which can be efficiently solved on low-end hardware. Thus, the map could be updated during the navigation. Unfortunately, these performances are limited to the 2D slice of the environment.

To include the 3D information efficiently, and help the downstream modules of the autonomous driving system (prediction, motion planning) to benefit from the localization and the mapping. The 3D occupancy grid map is encoded in an intermediate representation, such that this representation could be compressed, stored and loaded from the hard disk with few bits of memory, but in the same time this representation retains all the information necessary to do the localization and the global map update, this representation has to be robust to the perception uncertainty as well.

## 7.10 LIDAR-Based perception For Vehicle Localization in an HD Map

Participants: Farouk Ghallabi, Fawzi Nashashibi.

In the framework of the PhD thesis of Farouk Ghallabi (co-supervised by Renault S.A.) 29, we addressed the problem of accurate localization of autonomous vehicles on highway roads using LiDAR sensors and a highly accurate third party map. The proposed approach is based on two core modules: perception and map-matching. The perception module uses the 3D data enhanced by the LiDAR reflectivity to detect road primitive features: lane markings, barriers, traffic signs and guardrail reflectors. The map-matching module incorporates these measurements and aligns them against a highly accurate third party map. The map-matching is performed using a particle filter, which we have improved in order to deal with the particle deprivation problem. The proposed improvement uses the road geometry in order to optimize the spatial distribution of particles while maintaining the number of particles constant. To evaluate the proposed method, we compared the localization outputs of our system to a Global Navigation Satellite System (GNSS) with RTK corrections (ground truth). Experiments have been conducted on two highway roads. The first is an experimental test track of 5 km long located at Renault’s Aubevoye’s Technical Center. This track is designed to exactly replicate a two-lane highway environment. The second is a section of the A13 highway, running from Paris and ending at Aubevoye. The results are promising and show the feasibility of a localization system based on LiDARs alone and with a sparse map data representation.

## 7.11 Safe motion planning through occluded intersections

Participants: Renaud Poncelet, Anne Verroust-Blondet, Fawzi Nashashibi.

A safe motion planning is of great importance for autonomous driving. In urban environment, a number of dangerous events, such as the presence of moving vehicles in occluded areas have to be anticipated. For this purpose, we introduced a method computing a safe and confortable motion along a fixed path in an urban environment with dynamic vehicles that may be occluded. Our approach has been tested and validated on typical scenarios carried out on CARLA simulator (cf. 26 for more details) .

## 7.12 Youla-Kucera based multi-objective controllers: Application to autonomous vehicles

Participants: Imane Mahtout, Fawzi Nashashibi.

Proper autonomous driving requires vehicle stability, precise motion, and natural behavior guaranteeing comfort for passengers inside the vehicle. However, driving situations change depending on the road layout and potential interactions with other traffic agents. Furthermore, vehicle capabilities can be degraded because of the on-board sensors' limitations, or the complexity of the algorithm processing the perception data. Within the thesis of Imane Mahtout 31, multi-objective control architectures were proposed that can adapt the vehicle behavior to overcome the changes in the operating conditions and assure vehicle performance and stability. The automated control system is able to address any circumstances ranging from a sudden change in the driving situation (i.e. lane change, obstacle avoidance) to an inaccurate measurement. We used Youla-Kucera (YK) parametrization to design control structures that are able to recognize the driving situation changes, adapting the controller response to satisfy the required performance level and keeping the motion stability with a natural vehicle behavior. A novel control structures was proposed based on controller reconfiguration, improving both lateral and longitudinal control by solving the following problems: 1) The trade-off between precision in trajectory tracking and comfort when the driving situation changes in lateral motion; 2) The trade-off between robustness and performance when noise measurement appears in Adaptive Cruise Control (ACC) systems. The stability of the proposed controller is guaranteed thanks to YK parametrization.

The validation of the proposed control structures is provided in both simulation and real-time experimentation using a Renault ZOE vehicle. The adaptability of the controllers to autonomous driving tasks is proved in different operating conditions.

## 7.13 Cyberphysical constructs and mobile communications for fully automated networked vehicles

Participants: Gérard Le Lann.

Personal work in 2020 has been devoted to solving the following open problems:

• Fully automated crossing of unsignalized intersections of arbitrary topologies (any number of arteries), with switch times (next virtual green light turned on after virtual red light is turned off) in the order of a few milliseconds.
• UPFs (Unambiguous Physical Functions), simple variations of PUFs (Physically Unclonable Functions), for instantaneous, unambiguous, and privacy-preserving designation of a specific neighboring vehicle (no IP or MAC address). This is needed for engaging reciprocal communications and striking explicit agreements in the case of safety-critical maneuvers (e.g. lane changes, lateral insertions in vehicle strings, cuts of multilane traffic, zipper merging, etc.).
• How to eliminate exhaustion of short-term certificates (for authentication) and the need for importing new short-term certificates. With that solution, short-term certificates can be reused, while making physical tracking (routes followed are collected) and cyber tracking (espionage of communications) unfeasible.

Work with foreign partners in 2020 has been devoted to exploiting the CMX framework introduced in 6.

With a North-American company (San Diego and Boston), work has been focused on immunity to cyberattacks in networks of cohorts (an instance of cyberphysical constructs for SAE L5) and how to implement a proactive security module (Section 5 in 6).

With a Swedish University (Stockholm), work has been focused on pseudonymization and privacy properties with NGVs (Section 3 in 6).

## 7.14 Belief propagation inference for traffic prediction

Participant: Jean-Marc Lasgouttes.

This work 41, in collaboration with Cyril Furtlehner (TAU, Inria), deals with real-time prediction of traffic conditions in a urban setting with incomplete data. The main focus is on finding a good way to encode available information (flow, speed, counts,...) in a Markov Random Field, and to decode it in the form of real-time traffic reconstruction and prediction. Our approach relies in particular on the Gaussian belief propagation algorithm.

Through our collaboration with PTV Sistema, we obtained extensive results on large-scale datasets containing 250 to 2000 detectors. The results show very good ability to predict flow variables and a reasonably good performance on speed or occupancy variables. Some element of understanding of the observed performance are given by a careful analysis of the model, allowing to some extent to disentangle modelling bias from intrinsic noise of the traffic phenomena and its measurement process.

This year, our research report 41 has been reorganised and submitted to IEEE Transactions on Intelligent Transportation Systems.

## 7.15 Stabilization of traffic through cooperative autonomous vehicles

Participant: Guy Fayolle, Jean-Marc Lasgouttes.

We investigate in 33 the transfer function emanating from the linearization of a car-following model, when taking into account a driver reaction time. This leads to stability conditions, which are explicitly given. We also show how this reaction time can introduce a partial string instability. This is a joint work with Carlos Flores from UC Berkeley.

This is intended as a foundation of a larger work on traffic stabilization by means of a fleet of cooperative automated vehicles. However, contrary to some earlier work in, our approach is based on a car-following model with reaction-time delay, rather than on a first order fluid model. While the focus of the present work is on human-driven vehicles, the next steps will be to tackle shockwave analysis and adapted traffic-stabilizing control strategies.

This work has been submitted to IEEE Transactions on Automatic Control.

## 7.16 Automated vehicle collection using an hedonic game

Participants: Jean-Marc Lasgouttes.

This is a joint work with Mohamed Hadded (Vedecom) and Pascale Minet (EVA, Inria).

We consider a shared transportation system in an environment where human drivers collect vehicles that are no longer being used. Each driver, also called a platoon leader, is in charge of driving collected vehicles as a platoon to bring them back to some given location (e.g. an airport, a railway station). Platoon allocation and route planning for picking up and returning automated vehicles is one of the major issues of shared transportation systems that need to be addressed. We propose a coalition game approach to compute 1) the allocation of unused vehicles to a minimal number of platoons, 2) the optimized tour of each platoon and 3) the minimum energy consumed to collect all these vehicles.

In this coalition game, the players are the parked vehicles, and the coalitions are the platoons that are formed. This game, where each player joins the coalition that maximizes its payoff, converges to a stable solution. The quality of the solution obtained is evaluated with regard to three optimization criteria and its complexity is measured by the computation time required. Simulation experiments are carried out in various configurations. They show that this approach is very efficient to solve the multi-objective optimization problem considered, since it provides the optimal number of platoons in less than a second for 300 vehicles to be collected, and considerably outperforms other well-known optimization approaches like MOPSO (Multi-Objective Particle Swarm Optimization) and NSGA-II (Non dominated Sorting Genetic Algorithm).

This work has been accepted for publication in 2021 in Concurrency and Computation: Practice and Experience.

## 7.17 Genus and classification of Random walks in the quarter plane

Participants: Guy Fayolle.

In collaboration with R. Iasnogorodski (SPCPA, Saint-Petersburg), we analyze the kernel$K\left(x,y,t\right)$ of the basic functional equation associated with the tri-variate counting generating function (CGF) of walks in the quarter plane. In this short paper, taking $t\in \right]0,1\left[$, we provide the conditions on the step set {${p}_{i,j}$} to decide whether the walks are singular or regular, as defined in Section 2.3 of 2. These conditions are independent of $t\in \right]0,1\left[$ and given in terms of step set configurations. We also find the configurations for the kernel to be of genus 0, knowing that the genus is always $\le 1$. All these conditions are very similar to the case $t=1$ considered in 2. Our results extend the work 36, which considers only very special situations, namely when $t\in \right]0,1\left[$ is a transcendental number over the field $Q\left({p}_{i,j}\right)$.

## 7.18 Reflected brownian motion in a non convex cone

Participants: Guy Fayolle.

In this ongoing work, in collaboration with S. Franceschi (LMO, Paris-Saclay University) and K Raschel (CNRS, Tours University), we state a system of functional equations satisfied by the Laplace transform of the stationary distribution of a reflected Brownian motion (SRBM) in a two-dimensional non-convex cone. While the case of convex cones is now reasonably well studied, the framework of non-convex cones turns out to be more challenging, as shown by similar research carried out in a discrete setting. We show in particular that the problem can be reduced to a boundary value problem of Rieman-Hilbert-Carleman type on an hyperbola, for a two-dimensional vector of meromorphic functions. This seems to be a quite original result.

## 7.19 Random walks in orthants and lattice path combinatorics

Participants: Guy Fayolle.

In the second edition of the book 2, original methods were proposed to determine the invariant measure of random walks in the quarter plane with small jumps (size 1), the general solution being obtained via reduction to boundary value problems. Among other things, an important quantity, the so-called group of the walk, allows to deduce theoretical features about the nature of the solutions. In particular, when the order of the group is finite and the underlying algebraic curve is of genus 0 or 1, necessary and sufficient conditions have been given for the solution to be rational, algebraic or $D$-finite (i.e. satisfying a linear differential equation). In this framework, number of difficult open problems related to lattice path combinatorics are currently being explored, in collaboration with A. Bostan and F. Chyzak (project-team SPECFUN, Inria-Saclay), both from theoretical and computer algebra points of view: concrete computation of the criteria, utilization of differential Galois theory, genus greater than 1 (i.e. when some jumps are of size $\ge 2$), etc. A recent topic (mentioned in 2019) deals with the connections between simple product-form stochastic networks (so-called Jackson networks) and explicit solutions of functional equations for counting lattice walks, Some partial extensions of 32 are under development.

# 8 Bilateral contracts and grants with industry

## 8.1 Bilateral contracts with industry

Valeo Group: a very strong partnership is under reinforcement between Valeo and Inria. Several bilateral contracts were signed to conduct joint works on Driving Assistance, some of which Valeo is funding. This joint research includes:

• A CIFRE like PhD thesis is ongoing between Valeo and Inria (Maximilian JARITZ), dealing with multisensor processing and learning techniques for free navigable road detection.
• Valeo is currently a major financing partner of the “GAT” international Chaire/JointLab in which Inria is a partner. The other partners are: UC Berkeley, Shanghai Jiao-Tong University, EPFL, IFSTTAR, MPSA (Peugeot-Citroën) and SAFRAN.
• Technology transfer is also a major collaboration topic between RITS and Valeo as well as the development of a road automated prototype.
• Finally, Inria and Valeo are partners of the PIA French project CAMPUS ( Connected Automated Mobility Platform for Urban Sustainability) including SAFRAN, Invia and Gemalto. The aim of the project is the development of autonomous vehicles and the realization of two canonical uses-cases on highways and urban like environments.

Renault Group: Collaboration between Renault and RITS re-started in 2016. Different research teams in Renault are now working separately with RITS on different topics.

• A CIFRE like PhD thesis is ongoing between Renault and Inria (Farouk GHALLABI) The thesis deals with the accurate localization of an autonomous vehicle on a highway using mainly on-board low-cost perception sensors.
• Another CIFRE PhD thesis is ongoing since November 2017 (Imane MAHTOUT).

AKKA Technologies: Collaboration with AKKA since 2012 (for the Link & Go prototype).

• Inria and AKKA Technologies are partners in the VALET projects (ANR projects).
• A CIFRE PhD thesis (Luis ROLDAO) dealing with 3D-environment modeling for autonomous vehicles begun in October 2017.

# 9 Partnerships and cooperations

## 9.1 International initiatives

### 9.1.1 Inria international partners

#### Informal international partners

RITS has signed 3 MoU with the following international laboratories:

• Vehicle Dynamics and Control Laboratory, Seoul National University (SNU), S. Korea: international cooperation agreement for Graduate-Level Academic and Research Collaboration
• MICA Lab, Hanoi University of Science and Technology, Vietnam: cooperation agreement for research collaboration and PhD students co-supervision
• Integrated Industrial Design Lab (INDEL) of the Department of Product and Systems Design Engineering, University of the Aegean, Greece: international cooperation agreement for Graduate-Level Academic and Research Collaboration

### 9.1.2 Participation in other international programs

Samuel de Champlain Québec-France collaboration program called "Vision par ordinateur en conditions difficiles": cooperation between Raoul de Charette and Jean-François Lalonde from Laval University. In this context, several PhD students from Laval University visited RITS team this year (cf. next section) and Raoul de Charette is an associate researcher at Laval University.

This year, our collaborative work were published 19, 20 and two talks were given at Montreal AI Symposium 2020:

• T. Tremblay, S-S. Halder, R. de Charette, J-F. Lalonde - Rain rendering for evaluating and improving robustness to bad weather. Montreal AI Symposium 2020.
• E. Dubeau, M. Garon, B. Debaque, R. de Charette, J-F. Lalonde - RGB D E: Event Camera Calibration for Fast 6 DOF Object Tracking, Montreal AI Symposium 2020.

## 9.2 International research visitors

### 9.2.1 Internships

• Pranav Agarwal, until Jan 2020
• Di Ai, from Apr 2020 until Sep 2020
• Gabriel Faivre, from Mar 2020 until Aug 2020
• Manuel Gonzalez, until Feb 2020
• Clara Véré, SpirOps AI, until November 2020
• Leonardo Ward, until Feb 2020

### 9.2.2 Visits of international scientists

• Carlos Hidalgo, Tecnalia, Spain, until May 2020,
• Etienne Dubeau, Laval University, Canada, from February 3rd 2020 until March 31st 2020,
• Mathieu Garon, Laval University, Canada, from February 3rd 2020 until March 31st 2020,
• Maxime Tremblay, Laval University, Canada, until July 2020 (remote only)

### 9.2.3 Visits to international teams

Kaouther Messaoud was at UC San Diego, visiting Mohan Trivedi Lab, from January 17th 2020 to April 14th

## 9.3 European initiatives

### 9.3.1 Collaborations with major European organizations

• RITS is member of the euRobotics AISBL (Association Internationale Sans But Lucratif) and the Leader of “People transport” Topic. This makes from Inria one of the rare French robotics representatives at the European level. See also: http://www.eu-robotics.net/
• RITS is a full partner of VRA – Vehicle and Road Automation, a support action funded by the European Union to create a collaboration network of experts and stakeholders working on deployment of automated vehicles and its related infrastructure. VRA project is considered as the cooperation interface between EC funded projects, international relations and national activities on the topic of vehicle and road automation. It is financed by the European Commission DG CONNECT and coordinated by ERTICO – ITS Europe. See also: http://vra-net.eu/

## 9.4 National initiatives

### 9.4.1 ANR

#### VALET

• Title: Redistribution automatique d’une flotte de véhicules en partage et valet de parking
• Instrument: ANR
• Duration: January 2016 - September 2019
• Coordinator: Fawzi Nashashibi
• Partners: Inria, Ecole Centrale de Nantes (IRCCyN), AKKA Technologies
• Inria contact: Fawzi Nashashibi
• Abstract: The VALET project proposes a novel approach for solving car-sharing vehicles redistribution problem using vehicle platoons guided by professional drivers. An optimal routing algorithm is in charge of defining platoons drivers’ routes to the parking areas where the followers are parked in a complete automated mode. The main idea of VALET is to retrieve vehicles parked randomly on the urban parking network by users. These parking spaces may be in electric charging stations, parking for car sharing vehicles or in regular parking places. Once the vehicles are collected and guided in a platooning mode, the objective is then to guide them to their allocated parking area or to their respective parking lots. Then each vehicle is assigned a parking place into which it has to park in an automated mode.

#### Hianic

• Title: navigation autonome dans les foules inspirée par les humains (Human Inspired Autonomous Navigation In Crowds)
• Instrument: ANR
• Duration: January 2018 - December 2020
• Coordinator: Anne Spalanzani (Inria Rhône-Alpes, Chroma research team)
• Partners: Inria Rhône-Alpes, Inria Paris, LIG Laboratoire d'Informatique de Grenoble, LS2N - ECN Laboratoire des Sciences du Numérique de Nantes
• Inria contact: Fawzi Nashashibi
• Abstract: The HIANIC project will try to address some problems that will arise when these cars are mixed with pedestrians. The HIANIC project will develop new technologies in term of autonomous navigation in dense and human populated traffic. It will explore the complex problem of navigating autonomously in shared-space environments, where pedestrians and cars share the same environment.

Such a system will contribute both to urban safety and intelligent mobility in “shared spaces”. Negotiation will help to avoid frozen situations increasing the vehicle’s reactivity and optimizing the navigable space. Negotiation, Human-Aware Navigation and Communication will contribute to a better public acceptance of such autonomous systems and facilitate their penetration in the transportation landscape.

### 9.4.2 FUI

#### PAC V2X

• Title: Perception augmentée par coopération véhicule avec l'infrastructure routière
• Instrument: FUI
• Duration: September 2016 - December 2020
• Coordinator: SIGNATURE Group (SVMS)
• Partners: DigiMobee, LOGIROAD, MABEN PRODUCTS, SANEF, SVMS, VICI, Inria, VEDECOM
• Inria contact: Raoul de Charette
• Abstract: The objective of the project is to integrate two technologies currently being deployed in order to significantly increase the time for an automated vehicle to evolve autonomously on European road networks. It is the integration of technologies for the detection of fixed and mobile objects such as radars, lidars, cameras ... etc. And local telecommunication technologies for the development of ad hoc local networks as used in cooperative systems.

### 9.4.3 Competitivity Clusters

RITS team is a very active partner in the competitivity clusters, especially MOV'EO and System@tic. We are involved in several technical committees like the DAS SUR of MOV'EO for example.

RITS is also the main Inria contributor in the VEDECOM institute (IEED).

# 10 Dissemination

## 10.1 Promoting scientific activities

### 10.1.1 Scientific events: selection

#### Member of the conference program committees

• Fawzi Nashashibi, member of the conference program committee of: IEEE 16th International Conference on Intelligent Computer Communication and Processing (ICCP 2020), IEEE 16th International Conference on Control, Automation, Robotics and Vision (ICARCV'2020), 1st International Conference on Robotics, Computer vision and Intellligent Systems (ROBOVIS 2020)

#### Reviewer

• Raoul de Charette: Conference on Artificial Intelligence (AAAI 2020), British Machine Computer Vision Conference (BMVC 2020), Conference on Computer Vision and Pattern Recognition (CVPR 2020), European Conference on Computer Vision (ECCV 2020), Intelligent Transportation System Conference (ITSC 2020), Montreal AI Symposium (MAIS 2020)
• Jean-Marc Lasgouttes: Forum ISTS 2020.
• Fawzi Nashashibi: IEEE Intelligent Vehicles Symposium (IV 2020), IEEE Inteligent Trnsportation Systems Conference (ITSC 2020), International Conference on Control, Automation, Robotics and Vision (ICARCV 2020), International Conference on Robotics, Computer vision and Intellligent Systems (ROBOVIS 2020).
• Anne Verroust-Blondet: IEEE Intelligent Vehicles Symposium (IV 2020), IEEE Inteligent Trnsportation Systems Conference (ITSC 2020), International Conference on Control, Automation, Robotics and Vision (ICARCV 2020).

### 10.1.2 Journal

#### Member of the editorial boards

• Guy Fayolle: associate editor of the journal Markov Processes and Related Fields.
• Fawzi Nashashibi: associate editor of the journal IEEE Transactions on Intelligent Vehicles.
• Fawzi Nashashibi: associate editor of the journal IEEE Transactions on Intelligent Transportation Systems.
• Anne Verroust-Blondet: associate editor of the journal The Visual Computer.

#### Reviewer - reviewing activities

• Raoul de Charette: Transactions on Pattern Analysis and Machine Intelligence (TPAMI), IEEE Transactions on Intelligent Transportation Systems (T-ITS).
• Guy Fayolle: AAP, MPRF, PTRF, QUESTA, European Journal of Combinatorics, JSP, Physica A, Springer Science.
• Jean-Marc Lasgouttes: IEEE Transactions on Intelligent Transportation Systems.
• Fawzi Nashashibi: IEEE Transactions on Intelligent Transportation Systems, IEEE Transactions on Intelligent Vehicles, Transportation Research Part C, IEEE ACCESS, Journal of Traffic and Transportation Engineering JTTE.
• Anne Verroust-Blondet: IEEE Transactions on Intelligent Vehicles, IEEE Transactions on Intelligent Transportation Systems, The Visual Computer, Journal of Selected Topics in Signal Processing.

### 10.1.3 Invited talks

• Raoul de Charette: two talks at the workshop on Perception for Cooperative Driving, Paris / Virtual, 2020-12-08.
• Gérard Le Lann: online seminars for Volkswagen (Wolfsburg, Germany), a Chinese company CyberLabs, and French audiences (clubs of engineers, universities).
• Fawzi Nashashibi: keynote at the MACIF MOVIN’ON LAB – Atelier n°5."'Les données du véhicule autonome: Traitement, sauvegarde et sécurité des données"', September 3rd, 2020.

### 10.1.4 Scientific expertise

• Guy Fayolle is scientific advisor and associate researcher at the Robotics Laboratory of Mines ParisTech. He is also collaborating member of the research-team SPECFUN at Inria-Saclay.
• Jean-Marc Lasgouttes is member of the Inria Commission d'Evaluation of Inria.
• Jean-Marc Lasgouttes was a member of the 2020 CRCN recruiting commissions of Inria research centers Rennes – Bretagne Atlantique and Sophia Antipolis – Méditérannée.
• Gérard Le Lann: Contribution to White Book on Cybersecurity: current challenges and Inria's research directions, Section 6.3.2 on Robotics and connected autonomous vehicles, February 2019.
• Fawzi Nashashibi is an associate researcher at the Robotics Laboratory, Mines ParisTech. He is an evaluator/reviewer of European H2020 projects and of Spanish and Dutch projects on robotics and Intelligent Transportation Systems.
• Anne Verroust-Blondet: remote evaluator for European H2020 and ANR calls.

• Jean-Marc Lasgouttes is a member of the Comité Technique Inria and of the Comité Local Hygiène Sécurité et Conditions de Travail of Inria Paris.
• Guy Fayolle is a member of the working group IFIP WG 7.3.
• Fawzi Nashashibi is a member of the international Automated Highway Board Committee of the TRB (AHB30).

He is a member of the Board of Governors of the VEDECOM Institute representing Inria and of the Board of Governors of MOV'EO Competitiveness cluster representing Inria. He is also member of the DAS-SMI of MOV'EO cluster.

• Anne Verroust-Blondet is the scientific correspondent of the European affairs and of the International Partnerships for Inria Paris.

## 10.2 Teaching - Supervision - Juries

### 10.2.1 Teaching

• Licence: Fawzi Nashashibi, “Programmation avancée”, 84h, L1, Université Paris-8 Saint-Denis, France.
• Engineering, 2nd year: Fawzi Nashashibi, “Image synthesis and 3D Infographics”, 12h, M2, INT Télécom SudParis, IMA4503 “Virtual and augmented reality for autonomy”.
• Master: Jean-Marc Lasgouttes, “Analyse de données”, 54h, second year of Magistère de Finance (M1), University Paris 1 Panthéon Sorbonne, France.
• Master: Jean-Marc Lasgouttes, “Analyse de données”, 52.5h, Master 1 SIC APP, University Paris 1 Panthéon Sorbonne, France.
• Master: Fawzi Nashashibi, “Obstacle detection and Multisensor Fusion”, 4h, M2, INSA de Rouen.
• Master: Fawzi Nashashibi, “Perception and Image processing for Mobile Autonomous Systems”, 12h, M2, University of Evry.
• Mastere : Raoul de Charette, “Scene Understanding with Computer Vision”, 20h, post master, Mines ParisTech, France.
• Doctorat: Jean-Marc Lasgouttes, “Introduction au Boosting”, 10.5h, Mastère Spécialisé “Expert en sciences des données”, INSA-Rouen, France.

### 10.2.2 Supervision

• PhD: Farouk Ghallabi, “Precise self-localization of autonomous vehicles using lidar sensors and highly accurate digital maps on highway roads”, Mines ParisTech, PSL Research University, June 2020, supervisor: Fawzi Nashashibi.
• PhD: Maximilian Jaritz, “2D-3D scene understanding for autonomous driving”, Mines ParisTech, PSL Research University, June 2020, supervisor: Fawzi Nashashibi, co-supervisor: Raoul de Charette.
• PhD: Imane Matout, “Youla-Kucera based multi-objective controllers : Application to autonomous vehicles”, Mines ParisTech, PSL Research University, December 2020, supervisor: Fawzi Nashashibi, co-supervisor: Vicente Milanés.
• PhD in progress: Kaouther Messaoud, “Détermination des manoeuvres et des intentions des véhicules avoisinant un véhicule autonome”, UPMC Sorbonne University, October 2017, supervisor: Anne Verroust-Blondet, co-supervisors: Fawzi Nashashibi, Itheri Yahiaoui.
• PhD in progress: Fabio Pizzati, “Style transfer and domain adaptation for semantic segmentation, PSL Research University and University of Bologna, November 2019, co-supervisors: Andrea Prati, Stefano Selleri, Fawzi Nashashibi and Raoul de Charette.
• PhD in progress: Renaud Poncelet, “Navigation autonome en présence d'obstacles fortement dynamiques au mouvement incertain”, UPMC Sorbonne University, September 2018, supervisor: Anne Verroust-Blondet, co-supervisor: Fawzi Nashashibi.
• PhD in progress: Luis Roldao, “Modélisation 3D de l’environnement et de la manoeuvrabilité d’un véhicule”, UPMC Sorbonne University, October 2017, supervisor: Anne Verroust-Blondet, co-supervisor: Raoul de Charette.

### 10.2.3 Juries

• Raoul de Charette was a reviewer of the PhD thesis of Mrs. Wei Zhou - Analysing the Robustness of Semantic Segmentation for Autonomous Vehicles, The University of Sydney, June 4th, 2020.
• Fawzi Nashashibi was a reviewer of the mid-term PhD thesis of Mr. Haodi Zhang - Incremental Cross-Modality Deep Learning for Pedestrian Recognition, INSA de Rouen - Université de Rouen Normandie, July 08, 2020.
• Fawzi Nashashibi was a reviewer of the mid-term PhD thesis of Mr. Tristan Klempka - Connaissance de situation “cloud-based” pour des véhicules connectés en environnements dynamiques et imprévisibles, Université de Toulouse, July 20, 2020.
• Fawzi Nashashibi was a reviewer of the mid-term PhD thesis jury of Mrs. Ameci Chtourou - Contextual communication for intelligent transportation systems in hybrid networks, Université Paris-Saclay, September 20, 2020.
• Fawzi Nashashibi was a reviewer of the mid-term PhD thesis of Mr. Antoine Lima - Integrity of Information in a Cooperative Perception Network Applied to Autonomous Navigation, Sorbonne Université, October 16, 2020.
• Fawzi Nashashibi was a jury member of the PhD thesis of Mr. Ezequiel Gonzalez Debada - Motion Planning for Connected Automated Vehicles in Mixed Traffic: A Study of Roundabouts, EPFL, March 24, 2020.
• Fawzi Nashashibi was the President and a reviewer of the PhD thesis of Mr. Djotiname Kangbéni Konlambigue - Conception d’un système de localisation à l’intérieur de bâtiments par vision monoculaire embarquée, Université de Rouen Normandie, May 15, 2020.
• Fawzi Nashashibi was a reviewer of the PhD thesis of Mrs. Roxana Dia - Towards Environment Perception using Integer Arithmetic for Embedded Application, Université Grenoble Alpes, June 19, 2020.
• Fawzi Nashashibi was a reviewer of the mid-term PhD thesis of Mrs. Hind Laghmara - Multiple Object Detection and Association using Belief Theory. Application to Intelligent Vehicle Perception, Université de Haute Alsace, July 21, 2020.
• Fawzi Nashashibi was a reviewer of the mid-term PhD thesis of Mr. Mohamed Ouarch - Sciences informatiques et technologies tridimensionnelles au service de l'art illustratif et séquentiel, Université de Paris 8 Vincennes Saint-Denis, October 21, 2020.
• Fawzi Nashashibi was a reviewer of the PhD thesis of Mr. Cyrille Pierre - Localisation coopérative robuste de robots mobiles par mesure d’inter-distance, Université Clermont-Auvergne, December 16, 2020.

## 10.3 Popularization

### 10.3.1 Articles and contents

• Fawzi Nashashibi was interviewed by the following popularization journals:
• Les Echos, Journalist: Jacques HENNO, ”Pourquoi l'intelligence artificielle cale”, September 14, 2020.
• Techniques de l'ingénieur, Journalist: Pierre Thouverez. ”Les concepts de mobilités doivent s’adapter au contexte de chaque ville”, October 27, 2020.1
• Fawzi Nashashibi and Raoul de Charette were interviewed by Kevin Poireault for ”Véhicule autonome de l'échec au nouveau départ” published in IT Industrie & Technologies, October 20, 2020.2

### 10.3.2 Interventions

• Gérard Le Lann has co-authored "Les véhicules autonomes", which appeared in Blog Binaire Le Monde, July 17, 2020 3.

# 11 Scientific production

## 11.1 Major publications

• 1 inproceedings Z. Alsayed, G. Bresson, A. Verroust-Blondet and F. Nashashibi. '2D SLAM Correction Prediction in Large Scale Urban Environments'. ICRA 2018 - International Conference on Robotics and Automation 2018 Brisbane, Australia May 2018
• 2 bookG. Fayolle, R. Iasnogorodski and V. Malyshev. S. AsmussenP. GlynnY. Jan'Random Walks in the Quarter Plane: Algebraic Methods, Boundary Value Problems, Applications to Queueing Systems and Analytic Combinatorics'.40Probability Theory and Stochastic ModellingThe first edition was published in 1999Springer International PublishingFebruary 2017, 255
• 3 article D. Gonzalez Bautista, J. Pérez, V. Milanés and F. Nashashibi. 'A Review of Motion Planning Techniques for Automated Vehicles'. IEEE Transactions on Intelligent Transportation Systems April 2016
• 4 inproceedings S. Halder, J.-F. Lalonde and R. De Charette. 'Physics-Based Rendering for Improving Robustness to Rain'. ICCV 2019 - International Conference on Computer Vision ICCV 2019. Supplementary pdf / videos available on project page Seoul, South Korea October 2019
• 5 inproceedings M. Jaritz, T.-H. Vu, R. De Charette, E. Wirbel and P. Pérez. 'xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation'. Conference on Computer Vision and Pattern Recognition (CVPR) For a demo video, see http://tiny.cc/xmuda Virtual, United States June 2020
• 6 techreport G. Le Lann. 'Cyberphysical Constructs and Concepts for Fully Automated Networked Vehicles'. RR-9297 INRIA Paris-Rocquencourt October 2019
• 7 article I. Mahtout, F. Navas, V. Milanés and F. Nashashibi. 'Advances in Youla-Kucera parametrization: A Review'. Annual Reviews in Control June 2020
• 8 inproceedings K. Messaoud, I. Yahiaoui, A. Verroust-Blondet and F. Nashashibi. 'Non-local Social Pooling for Vehicle Trajectory Prediction'. Intelligent Vehicles Symposium (IV) Paris, France 2019
• 9 inproceedings F. Pizzati, P. Cerri and R. De Charette. 'Model-based occlusion disentanglement for image-to-image translation'. ECCV 2020 - European Conference on Computer Vision ECCV 2020 ECCV 2020 Glasgow / Virtual, United Kingdom August 2020
• 10 inproceedings L. Roldão, R. De Charette and A. Verroust-Blondet. 'LMSCNet: Lightweight Multiscale 3D Semantic Completion'. 3DV 2020 - International Virtual Conference on 3D Vision Accepted at 3DV 2020 (Oral). For a demo video, see http://tiny.cc/lmscnet. Code is available at https://github.com/cv-rits/LMSCNet Fukuoka / Virtual, Japan November 2020
• 11 article M. Tremblay, S. Halder, R. De Charette and J.-F. Lalonde. 'Rain Rendering for Evaluating and Improving Robustness to Bad Weather'. International Journal of Computer Vision 19 pages, 19 figures, IJCV 2020. September 2020

## 11.2 Publications of the year

### International journals

• 12 article G. Fayolle and R. Iasnogorodski. 'Conditions for some non stationary random walks in the quarter plane to be singular or of genus 0'. Markov Processes And Related Fields March 2021
• 13 article F. Garrido, L. González, V. Milanés, J. Pérez and F. Nashashibi. 'A Two-Stage Real-Time Path Planning : Application to the Overtaking Manuever'. IEEE Access July 2020
• 14 article I. Mahtout, F. Navas, V. Milanés and F. Nashashibi. 'Advances in Youla-Kucera parametrization: A Review'. Annual Reviews in Control June 2020
• 15 article K. Messaoud, I. Yahiaoui, A. Verroust-Blondet and F. Nashashibi. 'Attention Based Vehicle Trajectory Prediction'. IEEE Transactions on Intelligent Vehicles 2021
• 16 article J. Naranjo, F. Serradilla and F. Nashashibi. 'Speed Control Optimization for Autonomous Vehicles with Metaheuristics'. Electronics 9 4 March 2020
• 17 article F. Navas, V. Milanés, C. Flores and F. Nashashibi. 'Multi Model Adaptive Control for CACC applications'. IEEE Transactions on Intelligent Transportation Systems 2020
• 18 articleD.-V. Nguyen, T.-K. Dao, E. Castelli and F. Nashashibi. 'A Fusion Method for Localization of Intelligent Vehicles in Carparks'.IEEE Access82020, 99729 - 99739
• 19 article M. Tremblay, S. Halder, R. De Charette and J.-F. Lalonde. 'Rain Rendering for Evaluating and Improving Robustness to Bad Weather'. International Journal of Computer Vision September 2020

### International peer-reviewed conferences

• 20 inproceedings E. Dubeau, M. Garon, B. Debaque, R. De Charette and J.-F. Lalonde. 'RGB-D-E: Event Camera Calibration for Fast 6-DOF Object Tracking'. ISMAR 2020 - IEEE International Symposium on Mixed and Augmented Reality Virtual, Brazil November 2020
• 21 inproceedings D. González, F. Navas, I. Mahtout and V. Milanés. 'A first approach for a passenger-centered behavior on driverless vehicles'. MED'2020 - 28th Mediterranean Conference on Control and Automation Saint-Raphaël / Virtual, France September 2020
• 22 inproceedings M. Jaritz, T.-H. Vu, R. De Charette, E. Wirbel and P. Pérez. 'xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation'. Conference on Computer Vision and Pattern Recognition (CVPR) Virtual, United States June 2020
• 23 inproceedings K. Melbouci and F. Nashashibi. 'LPG-SLAM: a Light-weight Probabilistic Graph-based SLAM'. ICARCV 2020 - International Conference on Control, Automation, Robotics and Vision Shenzhen / Virtual, China https://www.icarcv.sg/ 2020
• 24 inproceedings F. Pizzati, P. Cerri and R. De Charette. 'Model-based occlusion disentanglement for image-to-image translation'. ECCV 2020 - European Conference on Computer Vision ECCV 2020 Glasgow / Virtual, United Kingdom August 2020
• 25 inproceedings F. Pizzati, R. De Charette, M. Zaccaria and P. Cerri. 'Domain Bridge for Unpaired Image-to-Image Translation and Unsupervised Domain Adaptation'. Winter Conference on Applications of Computer Vision (WACV ’20) WACV Snowmass, United States http://wacv20.wacv.net/ March 2020
• 26 inproceedings R. Poncelet, A. Verroust-Blondet and F. Nashashibi. 'Safe Geometric Speed Planning Approach for Autonomous Driving through Occluded Intersections'. ICARCV 2020 - 16th International Conference on Control, Automation, Robotics and Vision 16th International Conference on Control, Automation, Robotics and Vision Shenzhen, China December 2020
• 27 inproceedings L. Roldão, R. De Charette and A. Verroust-Blondet. 'LMSCNet: Lightweight Multiscale 3D Semantic Completion'. 3DV 2020 - International Virtual Conference on 3D Vision Fukuoka / Virtual, Japan November 2020

### Conferences without proceedings

• 28 inproceedings P. Agarwal, A. Betancourt, V. Panagiotou and N. Díaz-Rodríguez. 'Egoshots, an ego-vision life-logging dataset and semantic fidelity metric to evaluate diversity in image captioning models'. ICLR 2020 - 8th International Conference on Learning Representations Addis Ababa / Virtual, Ethiopia April 2020

### Doctoral dissertations and habilitation theses

• 29 thesis F. Ghallabi. 'Precise self-localization of autonomous vehicles using lidar sensors and highly accurate digital maps on highway roads'. Université Paris sciences et lettres June 2020
• 30 thesis M. Jaritz. '2D-3D scene understanding for autonomous driving'. Université Paris sciences et lettres June 2020
• 31 thesis I. Mahtout. 'Youla-Kucera based multi-objective controllers : Application to autonomous vehicles'. Université Paris sciences et lettres December 2020

### Reports & preprints

• 32 misc G. Fayolle. 'A note on the connection between product-form Jackson networks and counting lattice walks in the quarter plane'. January 2020
• 33 misc G. Fayolle, J.-M. Lasgouttes and C. Flores. 'Stability and String Stability of Car-following Models with Reaction-time Delay'. December 2020
• 34 misc M. Jaritz, T.-H. Vu, R. De Charette, É. Wirbel and P. Pérez. 'Cross-modal Learning for Domain Adaptation in 3D Semantic Segmentation'. February 2021

### Other scientific publications

• 35 misc K. Messaoud, N. Deo, M. Trivedi and F. Nashashibi. 'Trajectory Prediction for Autonomous Driving based on Multi-Head Attention with Joint Agent-Map Representation'. September 2020

## 11.3 Cited publications

• 36 misc T. Dreyfus, C. Hardouin, J. Roques and M. Singer. 'On the Kernel curves associated with walks in the quarter plane'. 2020
• 37 inproceedings G. Fayolle and C. Furtlehner. 'About Hydrodynamic Limit of Some Exclusion Processes via Functional Integration'. Int. Math. Conf. `50 Years of IPPI'' Proceedings on CD. ISBN 978-5-901158-15-9 Institute for Information Transmission Problems (Russian Academy of Sciences) Moscow July 2011
• 38 article G. Fayolle and R. Iasnogorodski. 'Random Walks in the Quarter-Plane: Advances in Explicit Criterions for the Finiteness of the Associated Group in the Genus 1 Case'. Markov Processes and Related Fields 21 4 Accepted for publication in the journal MPRF (Markov Processes and Related Fields) December 2015
• 39 articleG. Fayolle and J.-M. Lasgouttes. 'Asymptotics and Scalings for Large Product-Form Networks via the Central Limit Theorem'.Markov Processes and Related Fields221996, 317-348
• 40 inproceedings C. Furtlehner, Y. Han, J.-M. Lasgouttes, V. Martin, F. Marchal and F. Moutarde. 'Spatial and Temporal Analysis of Traffic States on Large Scale Networks'. 13th International IEEE Conference on Intelligent Transportation Systems ITSC'2010 Madère, Portugal September 2010
• 41 techreport C. Furtlehner, J.-M. Lasgouttes, A. Attanasi, L. Meschini and M. Pezzulla. 'Spatio-temporal Probabilistic Short-term Forecasting on Urban Networks'. RR-9236 INRIA Saclay, équipe TAU ; INRIA de Paris, équipe RITS ; PTV-SISTeMA December 2018
• 42 inproceedingsV. Martin, C. Furtlehner, Y. Han and J.-M. Lasgouttes. 'GMRF Estimation under Topological and Spectral Constraints'.7th European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases8725Lecture Notes in Computer ScienceNancy, FranceSpringer Berlin HeidelbergSeptember 2014, 370-385
• 43 articleV. Martin, J.-M. Lasgouttes and C. Furtlehner. 'Latent binary MRF for online reconstruction of large scale systems'.Annals of Mathematics and Artificial Intelligence7712016, 123-154