Section: Contracts and Grants with Industry

ANR SignCom

Participants : Stéphane Donikian [ contact ] , Sylvie Gibet, Franck Multon, Florian Nouviale.

SignCom is a 3-year project funded by the French National Agency for Research. SignCom involves academic and industrial partners: VALORIA lab. ( Project leader, University of Bretagne Sud, Vannes), IRIT lab. (Toulouse), M2S lab. (University of Rennes 2), Bunraku team (IRISA), and Polymorph Software firm (Rennes), WebSourd firm and IRIS association are external providers (Toulouse).

SignCom aims to improve the quality of the real-time interaction between real humans and virtual agents, by exploiting natural communication modalities such as gestures, facial expressions and gaze direction. Based on French Sign Language (FSL) gestures, the real and virtual humans produce statements towards their interlocutor through a dialog model.

The final objective of the project consists in elaborating two innovative multimodal interactive applications (using body, hand, gaze and face data), producing new ways of communication by recognizing FSL utterances, and synthesizing adequate responses with a 3D avatar. One of the recognition application, developed by the IRIT lab, is based on vision and uses a gaze and face tracking device (Facelab from Seeing machine). The other one, which is currently developed by the Bunraku team in the context of the reality center, uses a multimodal approach, involving full-body motion recognition through several data flows in real-time, including body motion data (using the ART tracking system), gaze and facial expression data (using the same Facelab system) and hand data (using the 5DT cybergloves). The synthesis is developed by the VALORIA lab. From recorded FSL sequences, multimodal data are retrieved from a dual-representation indexed database (semantic and raw data), and used to generate new FSL utterances.

For more information, see: http://www-valoria.univ-ubs.fr/signcom/ .