Architectures and Models for Interaction Group

Research topic of the group AMI: Ambient and Interaction

Y. Bellik, D. Béroule, A. Gharsellaoui, A. Mohamed, G. Pruvost, J.P. Sansonnet, B. Turner

Classical WIMP interaction models are not adequate within the context of ambient environments due to users' mobility, interaction devices heterogeneity, interaction context variability, etc. Hence, there is a need of new interaction models that will suit well users' needs in ambient environments.

Interaction Adaptation. A major issue of interaction in ambient environments concerns the interaction adaptation due to the highly dynamic variability of devices and physical environment properties. Having developed the WWHT model which allows adapting dynamically the presentation of information using different modalities, we started an analysis work to see if this model can be reused in the context of ambient environments. This analysis lead us to identify two significant limitations of the WWHT model: 1. The WWHT model could only manage output modalities; 2. the context information representation was too specific. To overcome these limitations, we have proposed two main extensions to the WWHT model. The first extension consists in adapting the model so that it becomes capable of handling interactions in both sides (input and output). Hence, the concept of Off-the-shelf Interaction Object (OIO) that represents pieces of interactive software able to provide interaction through different input and output modalities was introduced. The second extension concerned the representation of context information within the model. We decided to switch to an ontology based modelling of the interaction context. The use of ontologies allows the designer to write rules for user interaction adaptation in generic terms. This extended model has been used successfully in the ATRACO European project.

Merging Virtual and Physical Worlds. Another important issue in ambient environments concerns the definition of interaction models that allow the harmonious merging of the virtual and physical worlds. Thanks to collaboration with the "Conversational Agents" research topic of group CPU, we have integrated a virtual agent in our ambient platform (IRoom) and embedded it with location-aware capacities that allow the virtual agent to point at real objects of the real world. We conducted an experimental evaluation comparing this agent with an agent that does not perceive nor use the location of users and objects. The location-aware agent elicited higher levels of perceived presence and perceived adaptivity (See Illustration 3).

Ambient conversational modalities. We want also to explore interaction modalities that could improve in future ambient environments the psychological well-being of ordinary people. One of the major issues is the global acceptability of such systems, which raises the question of the kind of relationship a person can establish with her/his ambient, viewed as an intentional entity. Recent research on conversational agents has upheld their ability to facilitate the link between users and computer systems. In this context, we have envisioned the integration of such agents in ambient systems, along three main directions:

1. Assistant Conversational Agents: handle the function of assistance when it is explicitly requested by users, for example through questions put in natural language. The conversational situations are defined according to the role endorsed by the agent: presenter, teacher, coach etc.

2. Rational and Behavioral (R&B) agents: deal with the integration of psychological phenomena, such as personality traits, into assistant agents. In particular, we have proposed the R&B architecture for the generic implementation of the psychological behaviors of the FFM taxonomy in terms of influence operators over the rational decision making process of artificial agents.
   

3. Personification of ambient systems: for people to establish a closer relationship, it is necessary for an ambient to exhibit in its interactions, elements that could be interpreted by users as character cues. In this context, a first model about the expression of “ambient’s emotions and traits” through ambient output modalities has been proposed. Furthermore, we have defined a model for the natural language interaction between users and abstract topics (in relation with work in social informatics – see below) that focuses on the expression of emotions and traits in abstract entities. Aimed at the longer-term, a neurobiological model of Decision-Making has been designed, in which the system choices depend upon its “emotional” experience.
   

Social Informatics. Finally, from a social informatics point of view, ambient environments are places where people meet, get to know one another and decide to do things together collectively. We are looking specifically at two issues. The first concerns the emergence of a desire to cooperate with people that you’ve just met. For that, people generally have to understand what is expected of them. For example, in relation with the work on personification mentioned above, we have started to personify ambient systems by using agents to help clarify what is at stake in a given cognitive and social situation. However, even when people see the interest of cooperating, before doing so they often need assurance that they will be rewarded for investing their time and effort in trying to do things with others. This, then, is the second question: how is confidence in the solidity, mutual respect and reciprocity of social relationships built up? We consider that appropriate linguistic behaviour in a flow of conversation is crucial and are studying how words can trigger confidence reinforcement mechanisms in ambient systems.

An experiment conducted in the IRoom (Intelligent Room) in collaboration with the "Conversational Agents" research topic of group CPU. Users and objects are tracked thanks to the Ubisense location system. Users had to find some objects disseminated in the IRoom with the help of a virtual agent. Two experiment conditions were tested: 1- a virtual agent that does not perceive nor use the location of users and objects; 2- a location-aware virtual agent that adapts its spatial behavior to users’ and objects’ locations during the search task and who is capable of pointing at real objects in the real world.