This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Iba, S. Articles by Khosla, P. K. Search for Related Content Social Bookmarking                   What's this?

Interactive Multimodal Robot Programming

The Robotics Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213-3890, USA, iba{at}ri.cmu.edu

Christiaan J. J. Paredis

Systems Realization Laboratory, G. W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332-0405, USA, chris.paredis{at}me.gatech.edu

Pradeep K. Khosla

The Robotics Institute and Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213-3890, pkk{at}ece.cmu.edu

As robots enter the human environment and come into contact with inexperienced users, they need to be able to interact with users in a multimodal fashion—keyboard and mouse are no longer acceptable as the only input modalities. In this paper we introduce a novel approach for programming robots interactively through a multimodal interface. The key characteristic of this approach is that the user can provide feedback interactively at any time—during both the programming and the execution phase. The framework takes a three-step approach to the problem: multimodal recognition, intention interpretation, and prioritized task execution. The multimodal recognition module translates hand gestures and spontaneous speech into a structured symbolic data stream without abstracting away the user’s intent. The intention interpretation module selects the appropriate primitives to generate a task based on the user’s input, the system’s current state, and robot sensor data. Finally, the prioritized task execution module selects and executes skill primitives based on the system’s current state, sensor inputs, and prior tasks. The framework is demonstrated by interactively controlling and programming a vacuum-cleaning robot. The demonstrations are used to exemplify the interactive programming and the plan recognition aspect of the research.

Key Words: human-robot interaction • service robotics • robot planning

The International Journal of Robotics Research, Vol. 24, No. 1, 83-104 (2005)
DOI: 10.1177/0278364904049250


CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				D. Kulic, W. Takano, and Y. Nakamura Incremental Learning, Clustering and Hierarchy Formation of Whole Body Motion Patterns using Adaptive Hidden Markov Chains The International Journal of Robotics Research, July 1, 2008; 27(7): 761 - 784. [Abstract] [PDF]