This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Varshavskaya, P. Articles by Rus, D. Search for Related Content Social Bookmarking                         What's this?

Automated Design of Adaptive Controllers for Modular Robots using Reinforcement Learning

Computer Science and AI Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA, paulina{at}csail.mit.edu

Leslie Pack Kaelbling

Computer Science and AI Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA, lpk{at}csail.mit.edu

Daniela Rus

Computer Science and AI Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA, rus{at}csail.mit.edu

Designing distributed controllers for self-reconfiguring modular robots has been consistently challenging. We have developed a reinforcement learning approach which can be used both to automate controller design and to adapt robot behavior on-line. In this paper, we report on our study of reinforcement learning in the domain of self-reconfigurable modular robots: the underlying assumptions, the applicable algorithms and the issues of partial observability, large search spaces and local optima. We propose and validate experimentally in simulation a number of techniques designed to address these and other scalability issues that arise in applying machine learning to distributed systems such as modular robots. We discuss ways to make learning faster, more robust and amenable to on-line application by giving scaffolding to the learning agents in the form of policy representation, structured experience and additional information. With enough structure modular robots can run learning algorithms to both automate the generation of distributed controllers, and adapt to the changing environment and deliver on the self-organization promise with less interference from human designers, programmers and operators.

Key Words: learning and adaptive systems • cellular and modular robots • animation and simulation

The International Journal of Robotics Research, Vol. 27, No. 3-4, 505-526 (2008)
DOI: 10.1177/0278364907084983


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?