Not Logged In

Online Monte Carlo counterfactual regret minimization for search in imperfect information games

Online search in games has been a core interest of artificial intelligence. Search in imperfect information games (e.g., Poker, Bridge, Skat) is particularly challenging due to the complexities introduced by hidden information. In this paper, we present Online Outcome Sampling, an online search variant of Monte Carlo Counterfactual Regret Minimization, which preserves its convergence to Nash equilibrium. We show that OOS can overcome the problem of non-locality encountered by previous search algorithms and perform well against its worst-case opponents. We show that exploitability of the strategies played by OOS decreases as the amount of search time increases, and that preexisting Information Set Monte Carlo tree search (ISMCTS) can get more exploitable over time. In head-to-head play, OOS outperforms ISMCTS in games where non-locality plays a significant role, given a sufficient computation time per move.

Citation

V. Lisy, M. Lanctot, M. Bowling. "Online Monte Carlo counterfactual regret minimization for search in imperfect information games". Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), (ed: Gerhard Weiss, Pinar Yolum, Rafael H. Bordini, Edith Elkind), pp 27-36, May 2015.

Keywords:  
Category: In Conference
Web Links: ACM Digital Library

BibTeX

@incollection{Lisy+al:AAMAS15,
  author = {Viliam Lisy and Marc Lanctot and Michael Bowling},
  title = {Online Monte Carlo counterfactual regret minimization for search in
    imperfect information games},
  Editor = {Gerhard Weiss, Pinar Yolum, Rafael H. Bordini, Edith Elkind},
  Pages = {27-36},
  booktitle = {Joint Conference on Autonomous Agents and Multi-Agent Systems
    (AAMAS)},
  year = 2015,
}

Last Updated: October 29, 2020
Submitted by Sabina P

University of Alberta Logo AICML Logo