Overall Objectives
Scientific Foundations
Application Domains
New Results
Contracts and Grants with Industry
Other Grants and Activities

Section: New Results

Motion capture from video

Participants : Lionel Reveret, Laurent Favreau, Julien Diener, Christine Depraz, Marie-Paule Cani.

This research project aims at capturing motion from the automatic processing of video to provide information for 3D animation of characters, such as humans or animals. Unlike several approaches in computer vision research, the goal is not to recognize activities, but rather to acquire robust geometric hints to control animation. Three main projects are currently under investigation: facial animation, deformation of skin surface and motion of animals.

Automatic detection of animal footsteps from video

Participants : Laurent Favreau, Lionel Reveret.

In several domains of character animation, footsteps are one of the most important constraints. It guarantees one of the main aspects of a realistic animation of locomotion. This task, when done manually, is even more complex for quadrupeds. Being able to automatically predict the footsteps information from a video footage is thus an important contribution. The method developed is based on the design of a dedicated image filter to detect the pattern of animal legs. Along the time range of the video, the positive filter responses are clustered so that a single trajectory point is given per leg. As 2D images are considered (profile view), there exist ambiguities in the prediction of each individual foot position when side views of legs are crossing each other (typically left and right side of the animal, and front and back legs for higher velocities). A motion model has been developed to take into account this problem. This work has been done in collaboration of the University of Washington, in Seatle, USA.

Morphable model of quadruped skeletons for animating 3D animals

Participants : Lionel Reveret, Laurent Favreau, Christine Depraz, Marie-Paule Cani.

Skeletons are at the core of 3D character animation. The goal of this work is to design a morphable model of 3D skeleton for four footed animals, controlled by a few intuitive parameters. This model enables the automatic generation of an animation skeleton, ready for character rigging, from a few simple measurements performed on the mesh of the quadruped to animate (see fig. 18 ). Quadruped animals - usually mammals - share similar anatomical structures, but only a skilled animator can easily translate them into a simple skeleton convenient for animation. Our approach for constructing the morphable model thus builds on the statistical learning of reference skeletons designed by an expert animator. This raises the problems of coping with data that includes both translations and rotations, and of avoiding the accumulation of errors due to its hierarchical structure. Our solution relies on a quaternion representation for rotations and the use of a global frame for expressing the skeleton data. We then explore the dimensionality of the space of quadruped skeletons, which yields the extraction of three intuitive parameters for the morphable model, easily measurable on any 3D mesh of a quadruped. We evaluate our method by comparing the predicted skeletons with user-defined ones on one animal example that was not included into the learning database. We finally demonstrate the usability of the morphable skeleton model for animation. This work has been published at the EG/SIGGRAPH Symposium on Computer Animation, 2005  [29] .

Figure 18. Morphable model of skeletons

Motion capture of animal motion

Participants : Lionel Reveret, Laurent Favreau, Christine Depraz, Marie-Paule Cani.

The motion of animals is still a challenging problem in 3D animation, both for articulated motion and deformation of the skin and fur (see Figure  19 ). The goal of this project is to acquire information from the numerous video footage of wild animals. These animals are impossible to capture into a standard framework of motion capture with markers. There are several challenges in the usage of such video footage for 3D motion capture : only one 2D view is available, important changes occur in lighting, contrast is low between the animal and foreground, etc. Currently, a method has been developed to first extract a binary silhouette of the animals and then, to map this silhouette to pre-existing 3D models of animals and motion thanks to a statistical prediction. This work has been selected as one of the best paper of the Symposium on Computer Animation 2004 (SCA'04) and an extended version has been published to the Graphical Models Journal  [10] .

Figure 19. Motion capture of animal motion

Evaluation of 3D facial animation

Participant : Lionel Reveret.

Techniques for rendering of the skin surface have now achieved a highly realistic level. However, human subject are highly trained to perceive other faces and require 3D animation of faces to be accurate in terms of amplitude and timing of motion of facial features to be believable. We are conducting experiments with experimental psychologists to evaluate the naturalness of the synthetic control of facial features motion, using an exhaustive search over the tunings of the control parameters. Two rendering techniques are considered: a 3D geometrical modelling of the face including texture map (in collaboration with David Sander, from Experimental Department of the University of Geneva), and a 2D image-based approach using re-synthesis of video of real faces (in collaboration with Edouard Gentaz, from UFR de psychologie expérimentale, Université Pierre Mendès-France, Grenoble).


Logo Inria