Section: Scientific Foundations
Multimedia Documents Authoring
We are working on interactive authoring environments. Developing such environments is a challenging issue: structured multimedia documents are complex objects and the process of creating and updating them is complex too. Well-established paradigms for static office or technical documents do not work. The traditional WYSIWYG approach is useless in a context where the final form of the document (What You Get) is multiple and unknown at creation time. In addition, writing down the description of a document in some multimedia document language is extremely difficult, given the various levels of representation that are involved: content, logical structure, layout, style, synchronization, hypertext structure, navigation, dynamic behaviours, etc. New approaches are needed.
On the Web, multimedia documents are based on XML. They are considered through several types of structures: layout, time, navigation, animations. We are working on techniques that allow users to manipulate all these structures in homogeneous environments. The key idea is to present simultaneously several views of the document, each view showing a particular structure, and allowing the user to manipulate it directly. As the various structures of a document are not independant from each other, these views are ``synchronized'' to show the consequences of every change in all other views. The XML markup, although it can be accessed at any time, is handled by the tools, and the author does not have to worry about it.
Two editing tools based on this concept are under development, Amaya and LimSee. In Amaya the emphasis is put on the integration of several XML vocabularies and associated technologies, and on direct interaction with the Web: the user can edit remote documents in exactly the same way as local files. With LimSee the focus is on the time dimension of multimedia documents and their continuous media contents.
Authoring Models and Templates
Even with tools providing views for direct manipulation of various structures, the authoring task is often considered as too complex for most users because it requires a deep understanding of the semantics of the language (e.g. the SMIL timing model, or the most advanced features of XHTML). We are therefore working on a new authoring model for multimedia documents that allows the creation of generic or dedicated authoring tools with appropriate user-friendly GUI.
Our approach is first to focus on the logical structure of the document while keeping some semantics of proven technologies such as SMIL. The second core idea is to tightly integrate template definitions in this document model: the template is itself a document constrained by a schema-like syntax. The continuum between templates and documents permits to edit templates generically as any other document and within the same environment. It also allows a more natural authoring process where documents can be progressively created from existing templates up to a final state where all place-holders are filled and all options are decided: during all this process, the document has a status which is between a pure template and a completed instance.
The LimSee3 model based on these concepts is under development and will provide a generic platform for the development of dedicated authoring tools.
The same kind of model is implemented in Amaya, where it allows to cover the very wide variety of Web documents: institutional pages, technical reports, slide shows, curriculum vitae, address books, etc. The specific components of all these documents can be represented by the model in terms of lower-level languages such as XHTML. Authors can then handle documents in terms of these specific components while finally producing standard-conformant documents.
Editing Compound Documents through Databinding
Compounding by references means that documents using different languages (namespaces) are linked by references. This allows separate languages to work together, but implementations of the languages to be separated. Compound documents can be authored by a variety of means and we are interested in multimedia-centric authoring tools that can create time-based, interactive content.
We study the creation of these authoring tools through databinding. Databinding for compounding document formats by inclusion being still at the research level, we are considering databinding for compounding document formats by reference. Strongly typed references and access to different documents are well supported by the Eclipse Modeling Framework (EMF) schema compiler. This is a great advantage when it comes to build a graphical editing tool for compound documents and we have adopted it. We are specifically interested by problems related to events flow in a multi-document environment and by how different languages should cooperate in rendering to the screen and the auditory space.
Feedback on this research is given by constructing an editor for SVGT- iXMFT games on mobiles devices. iXMFT (interactive eXtensible Music Format Tiny) is a format we have designed for digital interactive audio on mobiles and for which a soundtrack manager can be built using JSR-234 audio engines.