Exponentiated Gradient Methods for Reinforcement Learning
- Doina Precup, McGill University, Montreal
- Richard S. Sutton, Department of Computing Science, University of Alberta
This paper introduces and evaluates a natural extension of linear exponentiated gradient methods that makes them applicable to reinforcement learning problems. Just as these methods speed up supervised learning, we find that they can also increase the efficiency of reinforcement learning. Comparisons are made with conventional reinforcement learning methods on two test problems using CMAC function approximators and replacing traces. On a small prediction task, exponentiated gradient methods showed no improvement, but on a larger control task (Mountain Car) they improved the learning speed by approximately 25%. A more detailed analysis suggests that the difference may be due to the distribution of irrelevant features.
Citation
D. Precup, R. Sutton. "Exponentiated Gradient Methods for Reinforcement Learning". International Conference on Machine Learning (ICML), Nashville, pp 272-277, July 1997.| Keywords: | conventional, distribution, linear, machine learning |
| Category: | In Conference |
BibTeX
@incollection{Precup+Sutton:ICML97,
author = {Doina Precup and Richard S. Sutton},
title = {Exponentiated Gradient Methods for Reinforcement Learning},
Pages = {272-277},
booktitle = {International Conference on Machine Learning (ICML)},
year = 1997,
}Last Updated: May 31, 2007Submitted by Staurt H. Johnson
