Hierarchical Policy Gradient Algorithms
Full Text: icml03.pdf
Hierarchical reinforcement learning is a gen- eral framework which attempts to acceler- ate policy learning in large domains. On the other hand, policy gradient reinforcement learning (PGRL) methods have received re- cent attention as a means to solve problems with continuous state spaces. However, they suffer from slow convergence. In this pa- per, we combine these two approaches and propose a family of hierarchical policy gradi- ent algorithms for problems with continuous state and/or action spaces. We also intro- duce a class of hierarchical hybrid algorithms, in which a group of subtasks, usually at the higher-levels of the hierarchy, are formulated as value function-based RL (VFRL) prob- lems and the others as PGRL problems. We demonstrate the performance of our proposed algorithms using a simple taxi-fuel problem and a complex continuous state and action ship steering domain.
Citation
M. Ghavamzadeh, S. Mahadevan. "Hierarchical Policy Gradient Algorithms". International Conference on Machine Learning (ICML), Washington, DC USA, pp 226-233, August 2003.| Keywords: | |
| Category: | In Conference |
BibTeX
@incollection{Ghavamzadeh+Mahadevan:ICML03,
author = {Mohammad Ghavamzadeh and Sridhar Mahadevan},
title = {Hierarchical Policy Gradient Algorithms},
Pages = {226-233},
booktitle = {International Conference on Machine Learning (ICML)},
year = 2003,
}Last Updated: June 11, 2007Submitted by Staurt H. Johnson
