Multi-Time Models for Temporally Abstract Planning
- Doina Precup, McGill University, Montreal
- Richard S. Sutton, Department of Computing Science, University of Alberta
Planning and learning at multiple levels of temporal abstraction is a key problem for artificial intelligence. In this paper we summarize an approach to this problem based on the mathematical framework of Markov decision processes and reinforcement learning. Current model-based reinforcement learning is based on one-step models that can not represent common-sense higher-level actions, such as going to lunch, grasping an object, or flying to Denver. This paper generalizes prior work on temporally abstract models (Sutton, 1995) and extends it from the prediction setting to include actions, control, and planning. We introduce a more general form of temporally abstract model, the multi-time model, and establish its suitability for planning and learning by virtue of its relationship to Bellman equations. This paper summarizes the theoretical framework of multi-time models and illustrates their potential advantages in a gridworld planning task.
Citation
D. Precup, R. Sutton. "Multi-Time Models for Temporally Abstract Planning". Neural Information Processing Systems (NIPS), Denver, CO, USA, pp 1050-1056, January 1998.| Keywords: | suitabilility, gridworld, higher-level, temporally, machine learning |
| Category: | In Conference |
BibTeX
@incollection{Precup+Sutton:NIPS98,
author = {Doina Precup and Richard S. Sutton},
title = {Multi-Time Models for Temporally Abstract Planning},
Pages = {1050-1056},
booktitle = {Neural Information Processing Systems (NIPS)},
year = 1998,
}Last Updated: May 31, 2007Submitted by Staurt H. Johnson
