Section: New Results

Advanced perception : Applications

Participants : Ezio Malis, Patrick Rives, Pascal Morin, Tiago Ferreira Goncalves, Melaine Gautier, José Raul Azinheira [ Univ. of Lisboa ] .

Visual servoing applied to aircraft automatic landing

Within the FP6 STReP European Project PEGASE, we were in charge of the development of a all-weather vision-based navigation aid system for the automatic approach and landing of fixed-wing aircraft. The interest for vision-based systems in aeronautics stems from the lack of alternatives to the standard ground-based instrument approach systems, like the ILS and the MLS, since the required onboard equipment is still not cost-effective for most general aviation. In addition, recent advances in terms of onboard infrared and millimetric-wave imagery sensors have improved the capability for pilots to execute non-precise approaches in all weather conditions. The objective of the developed work is thus to take advantage of these emergent imagery sensors in order to provide approach and automatic landing capabilities to the general aviation [37] .

The methods previously proposed in the literature rely essentially on geometric visual features like the side lines of the landing runway and the corresponding vanishing point with the drawback that the detection of these features only occurs for the last hundred meters of the approach. In order to cope with the absence of such geometric features at higher altitudes, we have proposed to track the region around the airport using the ESM dense visual tracker, by assuming the region-of-interest (ROI) as a planar, or quasi-planar, scene. The estimated pose of the aircraft with respect to the runway can thus be retrieved from the planar homography transformation between the current image of the onboard camera and georeferenced images, previously acquired during a learning flight. Here, the database of georeferenced images was chosen to be a set of equidistant images along the reference path mostly for two reasons: the low descent angle, typically , which excludes the use of georeferenced images acquired from the nadir direction, because of the important transformation required to match both images; but also the need for path planning in the image-based visual servoing (IBVS) scheme, for the aircraft to follow a pre-defined Cartesian trajectory. To avoid that the tracked region of the image leaves the camera's field-of-view (FOV) during positioning corrections, a dynamic management of ROI was also implemented and counterbalance a possible lack of texture in the image (blue sky and sea).

Figure 6. Dynamic management of ROIs under heavy rain: a) and b) the resize of the ROI to avoid that it goes out of FOV; and c) and d) the resize of the ROI selecting a well textured region of the image.

Concerning visual servoing algorithms the two usual control schemes were considered. In both cases, the reference trajectory is sampled with a set of images which constitutes successive desired inputs. For the position-based visual servoing (PBVS) scheme, the control objective is stated as a trajectory tracking problem in SE3. The position error is then computed from the decomposition of the homography between the current and the desired images. For the image-based visual servoing (IBVS) scheme, the control objective is stated as a trajectory tracking expressed in the SL3 homography set. The feedback error is directly built from the homography without any decomposition.

Figure 7. Visual tracking of the region around the Marseille Marignane Airport under rain and fog. The tracked ROI (left) corresponds to the patch (upper right) warped in order to match the template (bottom right).

Finally, we have proposed a modern design control technique based on the Linear Quadratic Regulator (LQR) will full state feedback, where the linearized model of the case-study aircraft was considered. In order to minimize the inherent coupling between dutch roll and roll modes, a lateral controller with two closed-loops was implemented. The inner-loop is based on an eigenstructure assignment technique that allows to assign not only the eigenvalues of the closed-loop dynamics but also the desired uncoupled eigenvectors. The outer-loop, based on the LQR, is thus in charge of the heading and lateral error control using a bank-to-turn approach.

Vehicle platooning :

The Tarot project addresses autonomy issues for military ground vehicles with a peculiar emphasis on platooning applications. A classical scenario is a convoy of vehicles going across a dangerous area (i.e. minefield) where each vehicle has to track the trajectory of the vehicle ahead perfectly. Such a task can be formulated as a visual tracking task and implemented using the ESM visual tracker presented above. First experiments have been carried out on the unmanned vehicle developped by Thalès equiped with a Pan-and-tilt unit (figure 8 ).

Figure 8. Platooning Task

Pedestrian tracking :

The aim of the Love project is to detect and track pedestrians seen by a camera mounted on a car. The trajectory of the pedestrian with respect to the car is estimated in order to decide wether the pedestrian may collide with the vehicle. We have used the ESM visual tracker presented above to estimate the pedestrian's motion . The experiments show that the tracker performs well when the aspect of the pedestrian does not change too much. We are currently addressing the tracking problem despite strong changes of the pedestrian's aspect and temporary occlusions.