Section: Scientific Foundations
Multimedia Models and Languages
Participants : Jacques Lemordant, Vincent Quint, Cécile Roisin, Irène Vatton.
We have participated in the international endeavor for defining a standard multimedia document format for the web that accommodates the constraints of different types of terminals. SMIL is the main outcome of this work. It focuses on a modular and scalable format that combines efficiently the different dimensions of a multimedia web document: synchronization, layout and linking. Our current work on multimedia formats follows the same trends.
Specific formats for audio are an important work item in the team. More specifically, we are participating in IASIG (Interactive Audio Special Interest Group), an international initiative for creating a new format for interactive audio called iXMF (Interactive eXtensible Music Format). We have defined an XML version of iXMF (without scripting, but with integrated 3D audio rendering) and we have implemented it in an audio engine adapted to embedded systems, based on OpenSL/ES (Open Sound Library for Embedded Systems).
Regarding discrete media in multimedia documents, popular document languages such as XHTML can represent a very broad range of documents, because they offer very general elements that can be used in many different situations. This advantage comes at the price of a very low level of representation, often considered as presentational. The concepts of microformats and semantic XHTML were developed to tackle this weakness. More recently, RDFa was introduced with the same goal. These formats add semantics to web pages while taking advantage of the existing XHTML infrastructure. This approach enables new applications that can be deployed smoothly on the web, but authors of web pages have very little help for creating and encoding this kind of semantic markup. A language that addresses these issues is developed and implemented in WAM. Called XTiger, its role is to specify semantically rich XML languages in terms of other, less expressive XML languages, such as XHTML.
Another multimedia authoring model, called the LimSee3 model, was developed in WAM, putting the emphasis on continuous media, such as video and sound, and on synchronization and layout issues. It is a component-oriented document model integrating homogeneously logical, spatial, and time structures in a language-neutral way. It is a generic document model from which more specialized models (called templates) can be derived for different classes of applications. This also makes it possible to create dedicated authoring tools from a generic platform implementing the generic model.
Whereas document formats represent a multimedia document with all its internal structures, description languages describe a document from outside and provide metadata. In the area of description languages for multimedia documents, significant standardization efforts have been spent, such as MPEG-7 for instance, but the problem is not solved yet. Many application domains cannot cope with the description languages available today. We have worked on this issue in cooperation with INA, the French national archive of broadcast radio and television. We have defined a structure description language for audio-visual documents, focusing on formal consistency to make descriptions usable in very large bases, such as those of INA. Typical applications of this work are the producion of a thematic audio-visual offer from archives, or the product ion of the same interactive application on various media (CD-ROM, DVD, web).
