Section: New Results

System architectures for pervasive computing

Participants : Michel Banâtre, Paul Couderc [ contact ] , Guillio Zecca, Minh Tuan Ho, Michele Dominici.

We are interested in two aspects in pervasive computing: the first is pervasive computing system support, where we study how generic mechanisms based on physical structures and processed can be proposed. The second is the support for collaborative capture.

Pervasive support at core system level

Pervasive computing involves tight links between real world activities and computing process. While the perception of the real world events can be handled entirely at the application, we think that ad hoc approaches have limitations, in particular the complexity and the difficulty to re-use the code between applications. Instead, we promote the use of system level abstraction that leverage on tangible structures and processes. Important properties of this approach is that applications are, by design, operating in an implicit way (“in the background” of physical processes). They also often exhibits simpler architectures, and “natural” scalability in the sense that being build upon existing real-world process, they are strongly destributed design that relies essentially on local interactions between physical entities.

In this general principle, we proposed operating system level mechanisms based on pervasive memory to ensure data resilience and coordination support for robotic application, which is our contribution to the Roboswarm EU project.

Resilience for robotic

A swarm of robots needs to plan the tasks to achieve. This planning is a complex process and, when a failure occurs, a replanning is needed [20] . A plan repair first aims at reducing the costs of the new plan by deleting tasks or task parts already performed. This plan repair needs the knowledge about past actions. Classical works like ALLIANCE [21] do this replanning in a collaborative way using wireless technologies that enable all robots to stay in contact with each other. Other works like [20] address the issue of replanning without investigating the problem of availability of data for the plan repair .

This knowledge about past actions is very easy to get when a global network is available and when all the entities of the system have a permanent access to this network (like in ALLIANCE). However, this network may be hard or impossible to set up. For example, 802.11 coverage on a large field needs for access points to relay informations between areas. The deployment of those access points can be expensive (time and money consuming). In dangerous area like exploration area (space, submarine) or fields on fire, this deployment is even impossible. Therefore, new techniques are needed.

In the context of Roboswarm, collaborative backup greatly helps when a robot fails. It can reduce the costs of a failure without assuming the availability of a global wireless infrastructure. The deployment of robots is then easier and faster. As the main aim of this system is to reduce costs of the replanning , this system must give priority to expensive task and data. This system can also decrease the swarm needs for a centralized entity (like a global server).

We proposed and implemented such system, built using the opportunistic communication paradigm. We have simulated this backup system for swarm robots on some applications (cleaning and exploration) and shown that it reduces costs in the case of failure without interference with the swarm process.

This system tends to improve the process even more in the case of less centralized systems (multiple tasks attributed to one entity with possibilities of dynamic task rescheduling).

Instead of using other robots as backup storage, a pervasive memory addressed using near field communication can also be considered: RFID type memory tags can be arranged in the environment to ensure data persistence independently of the robots.

Local coordination

We propose a collaborative algorithm for a swarm of robots that have to complete a cooperative task after spatial and temporal synchronisation using RFID landmarks. The contribution to the Roboswarm project is twofold: we address the problem of collaboration, showing that the robots can actually fulfil a common coordinated task; moreover, we included this algorithm in a scenario where no central communication infrastructure is available, proving that the swarm can smartly accomplish some given goals just relying on robot-to-robot communication. We find the rationale behind that solution very important both for a fallback in case the central system fails, and for different implementations and future development of our system. The symbolic task chosen as example provides for the synchronization of a subset of the swarm, that have to collectively push a piece of furniture, to allow the cleaning in that area.

Our work focuses on the utilisation of RFID tags only in the synchronisation zones, where they are needed to fulfil a cooperative task, which relies on the coordination of the robots both in space and in time. Partial hints about the task to be done, or announcements from team-mates, are used to lower the time spent on this search phase by the robots roaming to find some additional information. Once a robot reaches a free tag, it can create an association with it, meaning that it is the only one agent exploiting the information stored on that tag, and acting consequently. This is named the "capture" phase. When a member of the swarm finds a tag, it publishes the coordinates of the previous and of the next tag. Hence all the other robots can dynamically update the list of available tags, and go towards the nearest one from their current position. This strategy drastically reduces the number of collisions without a considerable increase in the distance to cover. To correct the accumulated odometric error, the robot resets its odometry sensors to the position stored in the tag. This little correction is an example of correct exploitation of the "intelligence" scattered in the environment.

Despite the complications that may arise, distributed approaches to task assignment are generally preferred in Multi-Agent Systems for several reasons, such as robustness to single agent failures, scalability of the system, time constraints of applications, constraints on communication load and computational power of the agents. Whenever an agent accomplishes a task, it announces its termination to the entire team, even though simultaneous tag discovery and asynchrony among messages can lead to some conflicts. During the development of our solution we considered a self-stimulation mechanism similar to the proposed one to invoke a group behaviour in all robots receiving that clue.

Coordination can be seen as divided into two levels: space, because robots must be in a pre-determined formation to trigger the start; time, because robots must synchronise to start together. The configuration seems to naturally rely on multi-hop message passing; however this was quite heavy and not so robust. Indeed, a lot of information must be exchanged to have updated and accurate data, and since the robots are mobile, we cannot waste resources, be they computation cycles, battery or others. Other strategies like gossip dissemination were neither so immediate nor so convenient to be implemented on simple robots that are unaware of the external world. The communication exploits thus a hybrid fashion between multi-hop and probabilistic flooding, so that each robot receiving an announce forwards it in its "broadcast bubble" described by its communication range and physical obstacles. This may not be the best strategy to optimise energy, but it is very probable that all the members will get the message, even the ones which are not in the coverage radius of the sender, and that are necessary for the outcome of the task (see Figure 5 ).

Figure 5. Robots (small circles) and their relative broadcast bubbles

When a robot captures a tag when no global communication infrastructure is provided it could be unaware of the state and activity of the other members of the swarm. It might have missed the announcements from its team-mates, so it could be the first one, an intermediate one or the last one discovering a synchronisation zone. The first thing to do is to communicate that a tag is there, and has been captured. In this way, the other robots who are still roaming will have a more accurate clue to find the target zone, and the ones that already captured a tag can announce their state to allow the upcoming robots to upgrade their vision of the world. The main problem of the algorithm is to have a synchronised trigger of the task, even with a minimal delay due to the goal type and the distributed nature of the task. The developed algorithm was conceived on the base of the Two Phase Commit protocol (2PC), quite used in distributed environments. In our case, the "ready to start" is given by the physical arrangement of the robots on the tags. Though all the robots act in the same way, one of them must take command for a while to grant a correct execution of the protocol.

To ensure that as many robots as possible participate in the collaboration, the leader waits for the capture of the other tags for a reasonable timeout. Afterwards, if at least the minimal number of robots are active on a tag, it broadcasts a message warning that the swarm is ready to start. Each robot receiving the announce retransmits the message, and checks which robots are on the previous and on the next tag, to create a kind of topology structure, needed afterwards for the control phase. When all the robots are ready, the leader commits the time to trigger the start.

We provided some simulations to ensure an opportune outcome. Finally, our proposal has been validated with qualitative and quantitative metrics on the distance run and the exchanged messages. The results showed that this kind of approach is practicable and that can be implemented in real life, and the implementation of such a system for operations like cleaning or equivalent is foreseen for the near future. The futuristic field of space exploration, together with the idea of setting up space colonies, is currently under research; the coordinated movement and the synchronisation among robots are definitely in scope.

Collaborative capture

The ubiquity of personal multimedia capture devices such as mobile phones brings new opportunities, such as the novel concept of collaborative or distributed capture [14] . Distributed capture consists in using a collection of personal devices dispersed throughout to capture data of an event or a reality as a distributed sensing infrastructure, and exchanging and synchronizing the resulting aggregate to increase the global quality.

An important challenge raised by this concept is to ensure coherency of the data being captured when merging fragments. For example, merging an audio stream captured by a device with a video captured by another raises temporal coherency issue [14] . Another example would be a swarm of robots collecting data using their onboard sensors, in order to make a map. This application raises spatial coherence issue as the spatial references of the data collected by an indivudal robot are dependant on sensor accuracy, such as robots odometry.

We are working on the problem of spatial coherence in the context of a collaborative photographic application, inline with the previous research. In such application, a mobile device can take pictures and reference the distance and the direction relatively in between the spots where the pictures are taken. In this way a map can be built of the environment wherein the photos are related by their distance and relative direction (angle). This information can be shared between users and this collectively built map can then serve as a photographic guide to others. One can expect the map would grow quite big easily, so the application should scale well to large collections. It makes sense to distribute the geotagged photo collection and any processing involved in locating and relating the pictures, so resources can be shared in a way that is achievable for limited mobile devices and without a central service.

A first approach was proposed to deal with spatial coherence issues that arise because of inaccuracies in the users estimates in this distributed capture example application. Distributed photo capture more specifically concerns the aggregation of photos taken with a mobile device, by several users and at different but related places, into a coherent collection. The coherence of the collection is clearly the quality metric here. A photo collection becomes spatially incoherent when a photo is added that is not at the location where the user expects it. Based on the virtual world model of the collection, the user might expect to see one thing but might get to see something quite different that does not match reality. The solution we proposed consists in a distributed algorithm that spread the errors by relaxation. [9]

Definition of a proactive service

Devices such as mobile phones are the main interaction point for mobile users. We studied the possibilities offered by these devices within the framework of a museum. Two classes of interactions can be considered:

• Visitor requesting information from the site: a visitor uses an interactive service to receive a list of recommended objects. It explicitly requests information from these objects in the museum. These interactions can be seen as "pull" type. The visitor carries a mobile device, offering him access to web-type information system in a contextual manner: requests include context descriptors such as current position or user profile that can influence the available information or its presentation.

• Visitor automatically receiving contextual information: a proactive service senses the environment and spontaneously adapts its behaviour to the local context. These interactions can be seen as "push" type. Typical sensing involves determining the profiles of near by visitors, allowing the dynamic computation of group profiles and enabling global adaptation of the environment. For example, if a room uses vocal explanations or shows short documentation movies, environment sensing enables automatic selection of the dominant language accordingly to the current visitors in the area.

Typical sensing involves determining the profiles of near by visitors, allowing the dynamic computation of group profiles and enabling global adaptation of the environment to the context. The notion of "context" considered here refers to a description of a physical situation in the museum; a physical situation may be described by low descriptors such as raw sensor values (GPS position, RFID, etc.), but these one are usually not directly relevant for an application. Context-awareness requires higher level description that can be used easily by service. For example, an application shouldn't have to do only position computation if the goal is to determine what is the language spoken by nearby people. Instead the application should be able to request data matching some criteria, including spatial properties if required.

We designed a context-sensing information diffusion system to detect and identify group of users (co-located in a given area of the museum), and to provide them with unsolicited messages like "conference invitation at 14pm, room 12", with a multi-language support, and multi-modal support (Text To Speech is automatically activated for people with visual disability).

Two aspects have been addressed:

• First the service requires determining visitor's context in a given spatial area of the museum. For that, two kind of information are required: visitor's localization and personal parameters like "spoken language" and "ability to see". Localization can be accessed using RFID reading, combined with WLAN connectivity. Personal parameters can be extracted directly from profile stored in mobile device. Of course collecting these data in museum environment is a privacy concern, as it potentially enables visitor tracking when associated with user's account. So we only use "anonymous attributes" of the user's profile such as "language spoken", information sent by device application is not associated with user's account.

• Second a provisioning strategy is decided to target visitors in an efficient way. By efficient, we mean allocating the collective resources (for example, monitors in the museum) to larger groups (the "major" spoken language), and using mobile device for smaller groups. For that, we use the majority rule: the collective devices is allocated to display message in "major" language, while messages in "minor" languages and messages "compliant with Text To Speech engine" for users with visual disability, are forwarded to smartphones.

Coupled Objects

The concept of coupled objects 3.3.3 is a simple yet powerful pervasive computing approach to check or ensure integrity properties in real-life. In can be used to check for lost objects, to avoid or reduce thieft, to provide integrity checking and accountability in logistic and so on.

We developed the Ubi-Check system, which allows to ensure that a set of physical objects remains complete as it enters or leaves controlled area. A particular application of the system finds its way for airport security: objects or tools that are not authorized in boarding area are still introduced by workers operating maintenance in these area. Ubi-Check can ensure that a set of objects that has entered the area is complete when it leaves the area. This application will be experimented at the Geneva airport.

We also developed and experimented several advanced prototype applications for the logistic domain. With the development of E-commerce, a difficult issue is rising: ensuring that a package that is delivered to its destination match the recipient expectation, and ensuring accountability in case of a problem. Currently, when a package is not complete once shipped, it is difficult to determine whether the fault was from the sender, the recipient, or at some level during the shipping. Coupled objects offers an elegant solution to this problem, mostly transparent to the information systems of the different parties involved in the process which is a major point: as the system relies on data structure self contained into the physical objects being transported, the integrity properties can be checked in an autonomous way. This system, Ubi Post, is currently the object of discussion with potential industrial users.