Notes
Planning and learning at multiple levels of temporal abstraction is a key problem for artificial intelligence. In this paper we summarize an approach to this problem based on the mathematical framework of Markov decision processes and reinforcement learning. Conventional model-based reinforcement learning uses primitive actions that last one time step and that can be modeled independently of the learning agent. These can be generalized to macro actions, multi-step actions specified by a arbitrary policy and a way of terminating. Macro actions generalize the classical notion of a macro operator in that they are closed loop, uncertain, and of variable duration. Macro actions are needed to represent common-sense higher-level actions such as going to lunch, grasping an object, or traveling to a distant city. This paper generalizes prior work on temporally abstract models (Sutton, 1995) and extends it from the prediction setting to include actions, control, and planning. We define a semantics of models of macro actions that guarantees the validity of planning using such models. This paper present new results in the theory of planning with macro actions and illustrates its potential advantages in a gridworld task.
Citation
D. Precup,
R. Sutton,
S. Singh.
"Notes".
National Conference on Artificial Intelligence (AAAI), Providence, Rhode Island, January 1997.
| Keywords: |
artificial intelligence, loop, macro actions, machine learning |
| Category: |
|
BibTeX
@incollection{Precup+al:AAAI97,
author = {Doina Precup and Richard S. Sutton and Satinder Singh},
title = {Notes},
booktitle = {National Conference on Artificial Intelligence (AAAI)},
year = 1997,
}Last Updated: April 24, 2007
Submitted by Christian Smith 
