Empirical Comparison of Gradient Descent and Exponentiated Gradient Descent in Supervised and Reinforcement Learning
- Doina Precup, McGill University, Montreal
- Richard S. Sutton, Department of Computing Science, University of Alberta
This report describes a series of results using the exponentiated gradient descent (EG) method recently proposed by Kivinen and Warmuth. Prior work is extended by comparing speed of learning on a nonstationary problem and on an extension to backpropagation networks. Most significantly, we present an extension of the EG method to temporal-difference and reinforcement learning. This extension is compared to conventional reinforcement learning methods on two test problems using CMAC function approximators and replace traces. On the larger of the two problems, the average loss was approximately 25% smaller for the EG method. The relative computational complexity and parameter sensitivity of the two methods is also discussed.
Citation
D. Precup, R. Sutton. "Empirical Comparison of Gradient Descent and Exponentiated Gradient Descent in Supervised and Reinforcement Learning". Technical Report, January 1996.Keywords: | exponentiated, CMAC function, machine learning |
Category: | Technical Report |
BibTeX
@manual{Precup+Sutton:96, author = {Doina Precup and Richard S. Sutton}, title = {Empirical Comparison of Gradient Descent and Exponentiated Gradient Descent in Supervised and Reinforcement Learning}, year = 1996, }Last Updated: May 31, 2007
Submitted by Staurt H. Johnson