Bayesian Actor-Critic Algorithms
Full Text: bac_icml07.pdf
We present a new actor-critic learning model in which a Bayesian class of non-parametric critics, using Gaussian process temporal difference learning is used. Such critics model the state-action value function as a Gaussian process, allowing Bayes' rule to be used in computing the posterior distribution over state-action value functions, conditioned on the observed data. Appropriate choices of the prior covariance (kernel) between state-action values and of the parametrization of the policy allow us to obtain closed-form expressions for the posterior distribution of the gradient of the average discounted return with respect to the policy parameters. The posterior mean, which serves as our estimate of the policy gradient, is used to update the policy, while the posterior covariance allows us to gauge the reliability of the update.
Citation
M. Ghavamzadeh, Y. Engel. " Bayesian Actor-Critic Algorithms". International Conference on Machine Learning (ICML), pp 297-304, June 2007.| Keywords: | machine learning |
| Category: | In Conference |
BibTeX
@incollection{Ghavamzadeh+Engel:ICML07,
author = {Mohammad Ghavamzadeh and Yaakov Engel},
title = { Bayesian Actor-Critic Algorithms},
Pages = {297-304},
booktitle = {International Conference on Machine Learning (ICML)},
year = 2007,
}Last Updated: February 01, 2008Submitted by Nelson Loyola
