Modeling Human-Robot-Interaction Based on Generic Interaction Patterns

2008-03 till 2012-06
Research Areas: 

While current techniques for human-robot interaction modeling are typically limited to restrictive command-control style, traditional dialog modeling approaches are not directly applicable to robotics due to the lack of real-world integration. We have introduced a new approach to dialog modeling on robots that includes (1) a task-state protocol providing a fine-grained interface to the complex domain processing of the robotic system and (2) the concept of generic interaction patterns that support rapid prototyping of human-robot interactions.


Methods and Research Questions: 

The idea of making robots a ubiquitous technology in order to help people with their every day tasks in home or office environments has imposed new challenges on interaction modeling in robotics.

The challenges in modeling human-robot-interaction are versatile. We have to deal with situated interaction in dynamic environments, involving mixed initiative and multiple modalities. Also, interactions are unstructured, potentially open-ended and may involve multiple tasks at a time.

We have suggested a technique for dialog modeling on robots that relies on a fine-grained task state protocol as interface between the dialog manager and system components. Typically, a task gets initiated, accepted, may be canceled or updated, may deliver intermediate results and finally is completed. Alternatively, it may be rejected by the handling component or execution may fail. Using this task state protocol, tight integration of perception and action into interaction can be achieved. For instance, a robot performing object manipulations might benefit from a human tutor supervising and correcting them. Given the update state, the human can repeatedly correct certain aspects of the action during execution, e.g. the position or the grip. Moreover, the task protocol supports interactive learning by establishing mechanisms for information negotiation between the dialog system and the responsible system component.

Furthermore, we have introduced the concept of interaction patterns that describe generic dialog strategies. Interaction patterns can be tailored with an application-specific configuration and combined in a flexible way, allowing for non-restrictive mixed-initiative interaction. The robot information request pattern, for instance, begins with the robot asking for the desired information, the human then gives the answer, the robot repeats it, and if necessary the human corrects it. By combining the above task states with robot dialog acts, the conversation level is related with the domain level. For instance, within the human command pattern, the robot acknowledges successful execution once the task is completed or apologizes if it has failed.

The patterns have been extracted from different human-robot interaction scenarios, such as a home-tour scenario in which a mobile robot acquires information about its environment, an object manipulation scenario in which a humanoid robot learns how to label and grasp objects, and a receptionist scenario in which a robot offers visitor guidance. Given the diversity of these scenarios, we argue that the resulting patterns provide a good coverage of typical situations in human-robot interaction.


The evaluation of a dialog framework must consider several aspects. As to the dialog strategy, we have demonstrated how mixed initiative can facilitate both the robot's learning process and the interaction itself. Recently, we have further investigated the role of the robot's task initiative with respect to the following questions: Which aspects of interaction have the greatest impact on user satisfaction? And what impact does the robot's task initiative have on objective success and the user's impression of it?

One aspect that is often neglected is framework usability. In order to evaluate the extent to which this approach enables developers to design new human-robot interaction scenarios, we conducted a usability test which showed that programmers unfamiliar with the dialog manager were able to build a simple interaction scenario consisting of begin and end of interaction and one person following task within one hour.

The presented approach has already been applied to several interaction scenarios e.g. the Home-Tour scenario, the Curious Robot scenario, a receptionist scenario, a multi-party quiz game with Nao, the current RoboCup@Home competition as well as the Citec dialog demonstrator. This demonstrates the generalizability of the approach.