Not Logged In

Empirical Comparison of Gradient Descent and Exponentiated Gradient Descent in Supervised and Reinforcement Learning

Full Text: empirical-comparison-of-gradient.pdf PDF

This report describes a series of results using the exponentiated gradient descent (EG) method recently proposed by Kivinen and Warmuth. Prior work is extended by comparing speed of learning on a nonstationary problem and on an extension to backpropagation networks. Most significantly, we present an extension of the EG method to temporal-difference and reinforcement learning. This extension is compared to conventional reinforcement learning methods on two test problems using CMAC function approximators and replace traces. On the larger of the two problems, the average loss was approximately 25% smaller for the EG method. The relative computational complexity and parameter sensitivity of the two methods is also discussed.

Citation

D. Precup, R. Sutton. "Empirical Comparison of Gradient Descent and Exponentiated Gradient Descent in Supervised and Reinforcement Learning". Technical Report, January 1996.

Keywords: exponentiated, CMAC function, machine learning
Category: Technical Report

BibTeX

@manual{Precup+Sutton:96,
  author = {Doina Precup and Richard S. Sutton},
  title = {Empirical Comparison of Gradient Descent and Exponentiated Gradient
    Descent in Supervised and Reinforcement Learning},
  year = 1996,
}

Last Updated: May 31, 2007
Submitted by Staurt H. Johnson

University of Alberta Logo AICML Logo