Not Logged In

Online implicit agent modelling

The traditional view of agent modelling is to infer the explicit parameters of another agent's strategy (i.e., their probability of taking each action in each situation). Unfortunately, in complex domains with high dimensional strategy spaces, modelling every parameter often requires a prohibitive number of observations. Furthermore, given a model of such a strategy, computing a response strategy that is robust to modelling error may be impractical to compute online. Instead, we propose an implicit modelling framework where agents aim to estimate the utility of a fixed portfolio of pre-computed strategies. Using the domain of heads-up limit Texas hold'em poker, this work describes an end-to-end approach for building an implicit modelling agent. We compute robust response strategies, show how to select strategies for the portfolio, and apply existing variance reduction and online learning techniques to dynamically adapt the agent's strategy to its opponent. We validate the approach by showing that our implicit modelling agent would have won the heads-up limit opponent exploitation event in the 2011 Annual Computer Poker Competition.

Citation

N. Bard, M. Johanson, N. Burch, M. Bowling. "Online implicit agent modelling". Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), (ed: Maria L. Gini, Onn Shehory, Takayuki Ito, Catholijn M. Jonker), pp 255-262, May 2013.

Keywords:  
Category: In Conference
Web Links: ACM Digital Library

BibTeX

@incollection{Bard+al:AAMAS13,
  author = {Nolan Bard and Michael Johanson and Neil Burch and Michael Bowling},
  title = {Online implicit agent modelling},
  Editor = {Maria L. Gini, Onn Shehory, Takayuki Ito, Catholijn M. Jonker},
  Pages = {255-262},
  booktitle = {Joint Conference on Autonomous Agents and Multi-Agent Systems
    (AAMAS)},
  year = 2013,
}

Last Updated: October 29, 2020
Submitted by Sabina P

University of Alberta Logo AICML Logo