Predictive Representations of State
- Michael L. Littman, AT&T Labs-Research, Florham Park, New Jersey
- Richard S. Sutton, Department of Computing Science, University of Alberta
- Satinder Singh, University of Michigan, Ann Arbor, MI
We show that states of a dynamical system can be usefully represented by multi-step, action-conditional predictions of future observations. State representations that are grounded in data in this way may be easier to learn, generalize better, and be less dependent on accurate prior models than, for example, POMDP state representations. Building on prior work by Jaeger and by Rivest and Schapire, in this paper we compare and contrast a linear specialization of the predictive approach with the state representations used in POMDP and in k-order Markov models. Ours is the first specific formulation of the predictive idea that includes both stochasticity and actions (controls). We show that any system has a linear predictive state representation with number of predictions no greater than the number of states in its minimal POMDP model.
Citation
M. Littman, R. Sutton, S. Singh. "Predictive Representations of State". Predictive Representations of World Knowledge, January 2002.| Keywords: | machine learning |
| Category: |
BibTeX
@incollection{Littman+al:PredictiveRepresentationsofWorldKnowledge02,
author = {Michael L. Littman and Richard S. Sutton and Satinder Singh},
title = {Predictive Representations of State},
booktitle = {Predictive Representations of World Knowledge},
year = 2002,
}Last Updated: March 07, 2007Submitted by Nelson Loyola
