This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Endo, G. Articles by Cheng, G. Search for Related Content Social Bookmarking                         What's this?

Learning CPG-based Biped Locomotion with a Policy Gradient Method: Application to a Humanoid Robot

Tokyo Institute of Technology 2-12-1 Ookayama, Meguro-ku Tokyo, 152-8550, Japan, gendo{at}sms.titech.ac.jp

Jun Morimoto

ATR Computational Neuroscience Laboratories Computational Brain Project, ICORP Japan Science and Technology Agency 2-2-2 Hikaridai, Seika-cho, Soraku-gun Kyoto, 619-0288, Japan, xmorimo{at}atr.jp

Takamitsu Matsubara

ATR Computational Neuroscience Laboratories 2-2-2 Hikaridai, Seika-cho, Soraku-gun Kyoto, 619-0288, Japan, takam-m{at}atr.jp, Nara Institute of Science and Technology 8916-5 Takayama-cho, Ikoma-shi Nara, 630-0192, Japan

Jun Nakanishi

ATR Computational Neuroscience Laboratories Computational Brain Project, ICORP Japan Science and Technology Agency 2-2-2 Hikaridai, Seika-cho, Soraku-gun Kyoto, 619-0288, Japan, jun{at}atr.jp

Gordon Cheng

ATR Computational Neuroscience Laboratories ICORP, Japan Science and Technology Agency 2-2-2 Hikaridai, Seika-cho, Soraku-gun Kyoto, 619-0288, Japan, gordon{at}atr.jp

In this paper we describe a learning framework for a central pattern generator (CPG)-based biped locomotion controller using a policy gradient method. Our goals in this study are to achieve CPG-based biped walking with a 3D hardware humanoid and to develop an efficient learning algorithm with CPG by reducing the dimensionality of the state space used for learning. We demonstrate that an appropriate feedback controller can be acquired within a few thousand trials by numerical simulations and the controller obtained in numerical simulation achieves stable walking with a physical robot in the real world. Numerical simulations and hardware experiments evaluate the walking velocity and stability. The results suggest that the learning algorithm is capable of adapting to environmental changes. Furthermore, we present an online learning scheme with an initial policy for a hardware robot to improve the controller within 200 iterations.

Key Words: humanoid robots • reinforcement learning • bipedal locomotion • central pattern generator

The International Journal of Robotics Research, Vol. 27, No. 2, 213-228 (2008)
DOI: 10.1177/0278364907084980


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?