Project : evasion
Section: New Results
Motion capture from video
This research project aims at capturing motion from the automatic processing of video to provide information for 3D animation of characters, such as humans or animals. Unlike several approaches in computer vision research, the goal is not to recognize activities, but rather to acquire robust geometric hints to control animation. Three main projects are currently under investigation: facial animation, deformation of skin surface and motion of animals.
Motion capture of facial expression
Previous results showed that 3D animation of facial expression can be described with a compact set of linear modes. This parametric reduction guarantees robustness when applied to the tracking of facial motion from video. However, it does not take into account the motion due to expressions such as smiling. The goal of this project is to combine several methods of facial animation into a coherent framework (see figure 14). As preliminary results, a method has been implemented to allow integration between linear models for speech production and pseudo-muscles models for expressions (see sections 7.3 and 7.4).
Motion capture of skin deformation
Participant : Lionel Reveret.
A character animation pipeline consists of 3 main phases: firstly, the character appearance is modelled as a 3D surface. Secondly, a simplified skeleton is built to animate several parts of the model. Finally, a skinning phase asserts how the geometry of a 3D model is precisely deformed by the underlying skeleton animation. For this phase, several semi-manual techniques exist. They take into account effects such as smoothness and bulge of muscles. These techniques are still tedious and time-consuming. As a goal to simplify this step, this project investigates if the motion of skin surface can be learned from video analysis. To ensure time coherence of tracked motion, we have designed and used a special cloth of elastic fabric, printed with black and white squares. Tracking from several cameras allow to rebuild the 3D surface. Later, the 3D surface will need to be related to rigid motion to be applicable to character animation.
Motion capture of animal motion
The motion of animals is still a challenging problem in 3D animation, both for articulated motion and deformation of the skin and fur (see figure 15). The goal of this project is to acquire information from the numerous video footage of wild animals. These animals are impossible to capture into a standard framework of motion capture with markers. There are several challenges in the usage of such video footage for 3D motion capture : only one 2D view is available, important changes occur in lighting, contrast is low between the animal and foreground, etc. Currently, a method has been developed to first extract a binary silhouette of the animals and then, to map this silhouette to pre-existing 3D models of animals and motion thanks to a statistical prediction.