Section: Scientific Foundations

3D cities reconstruction

Participants : Cédric Guiard, Chun Liu, Emilia Lombardo, André Gagalowicz.

Cities are of multi-dimensions, highly functional and with visual complexity. They are the concentration of history, culture, economy, and ecology, etc. Therefore, modeling and visualizing realistically 3D cities using computers covers various disciplines and is a real challenge to researchers in different research domains. The potential applications for creating 3D cities range from research and educational purposes like urban planning and global navigation to entertainment like tourism and 3D computer games. Buildings, as one of the main objects in cities, differ from one another in many aspects like motif, style, dimensions, etc. Facades can to a great extent reflect most aspects of one building. Therefore, our research will mainly focus on 3D facade reconstruction based on multi-source informations (including aerial images, ground images, architectural knowledge, etc.). Our goal is to create a real time 3D facade reconstruction system with a user friendly interface. Three major challenges will be tackled:

• our system will face not only expert users but also non-expert users; so, we have to conceive a user friendly interface. Is there a way to let all of our future users express their idea in 3D? This is a demanding task to provide our various potential users with easy-to-use interface that requires no/not much learning while keeping the high quality results.

• real-time and realism are two critical factors for 3D object modeling and visualization. Here, architectural knowledge, like floor height and width should be adopted in order to get plausible results, and meanwhile considering the application of our system, memory consuming and time consuming should be reduced.

• robustness is another important factor in facade reconstruction from multi-source information. Since the quality and complexity of the input information cannot perfectly meet our requirements (usually there are heavy image noise, measurement errors, small irregular elements, etc.), how to get stable and precise reconstruction result will be a tough nut to crack.

Ranging from the generation of typical buildings appearance considered to populate the environment to the accurate reconstruction of existing ones regarding specific fidelity and precision constraints, the production of 3D buildings representation draws a bridge between Computer Graphics and Computer Vision approaches.

Techniques in Computer Graphics are interested in large-scale modeling making use of procedural schemes. Introduced in the 70's [39] as a formal approach to architectural design, Shape Grammars have recently evolved to extend their derivation mechanisms ( [36] , [41] , [33] , [34] ). Shape grammars can be complemented by cellular textures [30] and generative mesh modeling to generate facade ornaments. Meanwhile; these techniques do not allow to account for precise similarity constraints to a real environment.

Conversely, there is a wide spectrum of strategies for urban reconstruction in Computer Vision. Depending on considered input data : single image [35] , ground-based facade images ( [21] , [40] ), interactive editing using aerial images [37] , aerial images combined with ground-based panorama images [42] , ground-based laser scans combined with aerial images or airborne laser scans [24] , [25] , these systems still resort to semi-automatic and notorious fragile methods having quite difficulties to extract the structural characterization of the considered constructions.

All these different acquisitions types (geo-referenced through the help of the numerical foot print; cadastre delivered by ING) are considered and accessible in the scope of the TERRA NUMERICA project. In this context, our scientific objective is to establish a virtuous loop linking image-based analysis techniques to model-based building characterization through the introduction of an innovative reconstruction scheme counting for the simultaneous registration of planimetric, 2D photo and 3D scans acquisitions. This approach is foreseen to couple the respective advantages carried by these techniques able to resolve their respective weaknesses in order to:

• Enlarge the space of the buildings representation that can be instantiated considering a procedural paradigm;

• Support an interactive parametrization and automate the reconstruction process;

• Integrate precision and fidelity constraints in data correspondence as a mean to incrementally qualify and enrich the model;

• Improve structural and semantic characterization under a hierarchical scheme.