Not Logged In

On the Significance of Markov Decision Processes

Formulating the problem facing an intelligent agent as a Markov decision process (MDP) is increasingly common in artificial intelligence, reinforcement learning, artificial life, and artificial neural networks. In this short paper we examine some of the reasons for the appeal of this framework. Foremost among these are its generality, simplicity, and emphasis on goal-directed interaction between the agent and its environment. MDPs may be becoming a common focal point for different approaches to understanding the mind. Finally, we speculate that this focus may be an enduring one insofar as many of the efforts to extend the MDP framework end up bringing a wider class of problems back within it.

Citation

R. Sutton. "On the Significance of Markov Decision Processes". Artificial Neural Networks - ICANN'97, (ed: W. Gerstner, A Germond, M. Hasler, J-D Nicoud), pp 273-282, January 1997.

Keywords: framework, goal-directed, artificial, machine learning
Category: In Journal

BibTeX

@article{Sutton:97,
  author = {Richard S. Sutton},
  title = {On the Significance of Markov Decision Processes},
  Editor = {W. Gerstner, A Germond, M. Hasler, J-D Nicoud},
  Pages = {273-282},
  journal = {Artificial Neural Networks - ICANN'97},
  year = 1997,
}

Last Updated: May 31, 2007
Submitted by Staurt H. Johnson

University of Alberta Logo AICML Logo