A Temporal-Difference Model of Classical Conditioning
- Richard S. Sutton, Department of Computing Science, University of Alberta
- Andrew Barto, Department of Computer Science, University of Massachusetts at Amherst
Rescorla and Wagner's model of classical conditioning has been one of the most influential and successful theories of this fundamental learning process. The learning rule of their theory was first described as a learning procedure for connectionist networks by Widrow and Hoff. In this paper we propose a similar confluence of psychological and engineering constraints. Sutton has recently argued that adaptive prediction methods called temporal-difference methods have advantages over other prediction methods for certain types of problems. Here we argue that temporal-difference methods can provide detailed accounts of aspects of classical conditioning behavior. We present a model of classical conditioning behavior that takes the form of a temporal-difference prediction method. We argue that it is an improvement over the Rescorla-Wagner model in its handling of within-trial temporal effects such as the ISI dependency, primacy effects, and the facilitation of remote associations in serial-compound conditioning. The new model is closely related to the model of classical conditioning that we proposed in 1981, but avoids some of the problems with that model recently identified by Moore et al. We suggest that the theory of adaptive prediction on which our model is based provides insight into the functionality of classical conditioning behavior.
Citation
R. Sutton,
A. Barto.
"A Temporal-Difference Model of Classical Conditioning". Conference of the Cognitive Science Society (CogSci), pp 355-378, January 1987.
| Keywords: |
Rescorla-Wagner, functionality, machine learning |
| Category: |
In Conference |
BibTeX
@incollection{Sutton+Barto:CogSci87,
author = {Richard S. Sutton and Andrew Barto},
title = {A Temporal-Difference Model of Classical Conditioning},
Pages = {355-378},
booktitle = {Conference of the Cognitive Science Society (CogSci)},
year = 1987,
}Last Updated: May 31, 2007
Submitted by Staurt H. Johnson 
