Section: New Results

Analysis and Exploitation of the reflectance properties and lighting

Image-based modelling exploiting the surface normal vector field and the visibility

We have developed multiview 3D reconstruction algorithms (recovering the 3D shape and the reflectance of a scene surface) which explicitly exploit the visibility and the properties of the surface normals. Our approach thus allows to naturally combine stereo, silhouette and shading cues in a single framework. This method applies to a number of classical scenarios - classical stereovision, multiview photometric stereo, and multiview shape from shading [23] . This method which has been first developed in level-set framework [20] , [46] , has then been adapted to mesh representations which provide more accurate results and more robust and more stable algorithms. This last work has been submitted to the International Journal of Computer Vision in october 2009. In this framework, we are also working on several specific applications and scenarios such as Helmholtz Stereopsis and dynamic photometric stereo. This work is still in progress.

Shape from ambient shading

We study the mathematical and numerical aspects of the estimation of the 3-D shape of a Lambertian scene seen under diffuse illumination. This problem is known as “shape from ambient shading” (SFAS), and its solution consists of integrating a strongly non-local and non-linear Integro-Partial Differential Equation (I-PDE). We provide a first analysis of this global I-PDE, whereas previous work had focused on a local version that ignored effects such as occlusion of the light field. We also design an original approximation scheme which, following Barles and Souganidis' theory, ensures the correctness of the numerical approximations, and discuss about some numerical issues. For more details, see our SSVM'09 paper [42] . This work is still in progress.

Finding Images with Similar Lighting Conditions in Large Photo Collections.

In our previous works on the analysis and exploitation of reflectance properties and lighting, the goal was to obtain accurate 3D surface and reflectance models of objects. This required controlled image acquisition conditions. We have recently started another line of work concerning lighting and reflectance, for less controlled settings. Concretely, like in the PhotoSynth system by Microsoft, we wish to exploit the large amount and variety of photographs available for free on the internet, via commodity collection such as flickR. For many major monuments, it is easy to download hundreds or thousands of images. Whereas PhotoSynth exploits them to generate 3D information from the images, we intend to use them to tell as much as possible about the appearance of objects. Appearance depends on reflectance properties and the lighting – in the above photo collections, we usually find photos representative for many different lighting conditions. If we are able to extract models for the appearance of objects, they may be used for various applications, e.g. relighting, without necessarily having to handle and estimate detailed physics-based reflectance models. Our first work along these lines consists in developing methods for clustering photographs into sets of images with similar lighting conditions [30] , [50] . To do so, we automatically segment the sky region in images and analyze the distribution of colors within it, leading to a clustering based on histograms of colors.

Multiple-View Geometry of the Refractive Plane.

Another new line of work concerns the analysis of images of semi-transparent objects. Our first result concerns the basic case of a planar refractive surface [28] . We have shown that images taken through such a surface, inspite the refractions, still have an epipolar geometry, and have shown how to formulate it analytically, via a fundamental matrix. Methods for estimating this and the relative position of cameras, have been proposed. Future work will be directed towards modeling of non-planar refractive semi-transparent surfaces.

Reconstructing specular surfaces.

In joint work with Jingyi Yu and Yuanyuan Ding from Delaware University, we have proposed a new method for reconstructing specular surfaces in 3D, from images [33] . The method exploits the fact that locally, specular surface patches act as reflectors that map 3D lines into conic curves on camera images. Observed conic curves reveal information about the specular surface, which is exploited here in a way analoguous to calibration of catadioptric cameras, albeit for more complicated shapes than the usual surfaces of revolution used for catadioptric cameras.