Section: New Results
Synoptic objects allowing autonomous agents to interact with a virtual environment
We propose the STARFISH (Synoptic-objects for Tracking Actions Received From Interactive Surfaces and Humanoids) architecture that uses Synoptic Objects to allow real-time object manipulation by autonomous agents in an informed environment. We define a minimal set of primitive Basic Actions which are used to build Complex Actions. We then assign these actions to Interactive Surfaces which are the parts of an object's geometry that are concerned by the action. The agent then uses these Interactive Surfaces to get the data specific to the object when it wants to manipulate it and to adapt its behavior accordingly.
STARFISHstands for Synoptic-objects for Tracking Actions Received From Interactive Surfaces and Humanoids. It is a new system which allows the easy definition of interactions between autonomous agents and the Synoptic Objects in the environment.
Synoptic Objectsare objects designed to offer to the autonomous agent a summary, or synopsis, of what interactions they afford. When an agent queries such an object, it knows what actions it can perform, where it should position itself, where it should place its hands, what state the object is in, whether it is allowed to perform the action, etc... All these indications, are given through the use of STARFISH Actions and Interactive Surfaces. An autonomous agent is itself considered as a Synoptic Object which allows interaction between agents (such as shaking hands) without any additional special considerations.
STARFISH Actionsconsist of a group of simple atomic actions, the Basic Actions which are the building blocks used to create Complex Actions. Through the Basic Actions described in Figure 30 , we can build more complex and varied actions. These Basic Actions are inspired by the Conceptual Dependency Theory (CDT), which seeks to represent complex natural language input using simple basic action verbs. We took CDT and used it the other way around: to generate complex behaviors using simple basic actions as building blocks. Note that for the remainder of this section, whenever we refer to a Basic Action, its name will be typeset in bold faceto differentiate it from the corresponding verb.
For example, the Open Door Complex Action (illustrated in figure 31 (a)) can be easily decomposed into its Basic Actions:
Transferself to the door.
Movearm towards knob.
Movehand to turn knob.
Displacedoor into the open position.
un- Graspto let go of the knob.
Interactive Surfacesare then used to model and describe the affordances of an object. They are generally parts of the object's geometry that act as hotspots when a certain action is to be accomplished, such as where to place the agent's hand when Grasping the doorknob or where to stand in order to open that door(Figure 32 ). Each Synoptic Object can have as many Interactive Surfaces as needed, depending on how much interaction it offers. Furthermore, many Interactive Surfaces of a single object can be associated to the same Basic Action, and the choice of the Interactive Surface is determined by the Complex Action the agent is doing. For example, in order to carry a suitcase, the agent has to Grabit by the handle, but in order to open the same suitcase, the agent has to Grabit by the lock.
Seeing how the Interactive Surfaces and STARFISH Actions are two completely independent concepts, we do not have to go through with them both when creating a new Synoptic Object. When an object with a new shape but with the same functionality has to be introduced into the simulation, it is only necessary to model its appropriate Interactive Surfaces without having to modify their associated behaviors.