Section: New Results
Visual tracking
Localization for augmented reality
Participants : Fabien Servant, Jean Laneurit, Éric Marchand.
This study focuses on real-time augmented reality for mobile devices. It is related to the France Telecom contract presented in Section 7.1 . The goal of this project is to enable augmented reality on mobile devices like GSM or PDA used by pedestrians in urban environments. With a camera and other external sensors, the absolute pose of the camera has to be computed in real-time to show to the end-user geolocalized information in an explicit way.
We have proposed a method for camera pose tracking that uses a partial knowledge on the scene. The method is based on a monocular vision localization method that uses previously known information about the environment (that is, the map of walls) and takes advantages from the various available databases and blueprints to constrain the problem. We have extended this approach in order to consider both a camera and an inertial sensor (IMU)[16] .
Other approaches, based on the definition of a differential tracker, have been applied in a museum environment (within the ANR Gamme project, see Section 8.2.3 ). Automatic recognition techniques adapted to such environments have thus been also developed to bootstrap the algorithm. A Kalman filter over SE(3) has also been developed in order to provide a better tracking and the capability to merge vision information with IMU information.
Robust model-based tracking for aircraft localization
Participants : Xiang Wang, Éric Marchand.
This work was realized within the European FP6 Pegase project (see Section 8.3.1 ). Our goal was to adapt the 3D model-based tracking algorithm Markerless [5] in order to allow localizing an aircraft. For that, we have considered a vectorial database of the surrounding of an airport, provided by Dassault Aviation. The method has been integrated in the Pegase simulation framework for a landing scenario on the Marignane airport starting more than 70 km from the airport. This year we also considered this approach on real aerial infrared images provided by Thales Optronic.
Robust tracking for controlling small helicopters
Participants : Céline Teulière, Éric Marchand.
The motivation of this work is to develop tracking algorithms that are suitable for the control of small UAV (X4 flyers). In the work carried out this year, the model-based tracking problem has been considered. Assuming a 3D model of the edges of an object is known, the tracking then consists in finding the camera pose which best aligns the projection of this model with the edges of the image.
The existing deterministic approaches (virtual visual servoing, Newton's minimization,...) usually suffer from possible ambiguities when tracking edges, since different edges may show very similar appearances leading to tracking errors. In order to handle these ambiguities, an optimisation method has been designed in which several hypotheses are maintained at the edge-tracking level, to retrieve several possible camera poses. This process is then used to optimize the best particles of a particle filter. Particle filtering framework allows the tracking to be robust to occlusions and large displacements that are expected in the considered application.
Tracking experiments have been conducted using image sequences from the embedded camera of the X4 flyer developed by CEA-List, and will soon be tested online. Current work aims at fusing inertial data from the UAV with the visual tracking, to improve the prediction of the filter and build an estimate of the UAV's velocity.
Omnidirectional stereovision
Participant : Éric Marchand.
This study is a joint work with Guillaume Caron and El Mustapha Mouaddib from MIS lab at the Université Jules Verne in Amiens. The motivation of this work is to take advantage of both the wide field of view induced by catadioptric cameras and of the redundancy brought by stereovision. Merging these two characteristics in a single sensor is obtained by combining a single camera and multiple mirrors. Within this framework we proposed a method to calibrate the stereo-catadioptric sensor. We also proposed a 3D model-tracking algorithm that allows a robust tracking of 3D objects using stereo catadioptric images given by this sensor. This work relies on an adapted virtual visual servoing approach, a non-linear pose computation technique. The model takes into account central projection and multiple mirrors. Results show robustness in illumination changes, mistracking and even higher robustness with four mirrors than with two [27] .
Objects tracking from mutual information
Participants : Amaury Dame, Éric Marchand.
This study originally focuses on rigid object tracking in non-structured environments. One of the main problem of tracking in outdoor environment is to deal with occlusions and illumination variations. The method used is based on a differential motion estimation. As in Section 6.1.3 , to be robust to changes of illuminations and occlusions, our approach is to use information of the image (as defined by Shannon) instead of using directly luminences. A metric derived from information theory, mutual information, is considered.
This metric has first been considered to replace the classical SSD tracker within a KLT-like tracker [35] . Within this context we also proposed a new way to extract interest points that are features to be tracked using this criterion.
Since mutual information is insensitive to changes in the lighting condition and to a wide class of non-linear image transformation, it is widely used in multi-modal image registration and more recently in motion estimation. This work shows that considering second order derivatives of mutual information allows reaching a better convergence frequency and a better accuracy. Experiments on planar object tracking and on 3D localization have been realized that validate this approach with respect to illumination changes and multi-modal images.
