Not Logged In

PapersDB

Monte Carlo tree search in continuous action spaces with execution uncertainty

• Timothy Yee
• Viliam Lisy
• Michael Bowling, University of Alberta

Real world applications of artificial intelligence often require agents to sequentially choose actions from continuous action spaces with execution uncertainty. When good actions are sparse, domain knowledge is often used to identify a discrete set of promising actions. These actions and their uncertain effects are typically evaluated using a recursive search procedure. The reduction of the problem to a discrete search problem causes severe limitations, notably, not exploiting all of the sampled outcomes when evaluating actions, and not using outcomes to help find new actions outside the original set. We propose a new Monte Carlo tree search (MCTS) algorithm specifically designed for exploiting an execution model in this setting. Using kernel regression, it generalizes the information about action quality between actions and to unexplored parts of the action space. In a high fidelity simulator of the olympic sport of curling, we show that this approach significantly outperforms existing MCTS methods.

Citation

BibTeX

@incollection{Yee+al:IJCAI16,
  author = {Timothy Yee and Viliam Lisy and Michael Bowling},
  title = {Monte Carlo tree search in continuous action spaces with execution
    uncertainty},
  Editor = {Subbarao Kambhampati},
  Pages = {690-697},
  booktitle = {International Joint Conference on Artificial Intelligence
    (IJCAI)},
  year = 2016,
}

Last Updated: October 28, 2020
Submitted by Sabina P

• Home
• Search
  • Advanced Search
• Publications
  • View Publication Entries
• Authors
  • View Authors
• User
  • Login or Register