Scaling Reinforcement Learning Toward RoboCup Soccer
- Peter Stone, Department of Computer Sciences, The University of Texas at Austin
- Richard S. Sutton, Department of Computing Science, University of Alberta
RoboCup simulated soccer presents many challenges to reinforcement learning methods, including a large state space, hidden and uncertain state, multiple agents, and long and variable delays in the effects of actions. We describe our application of episodic SMDP Sarsa(lambda) with linear tile-coding function approximation and variable lambda to learning higher-level decisions in a keepaway subtask of RoboCup soccer. In keepaway, one team, 'the keepers,' tries to keep control of the ball for as long as possible despite the efforts of 'the takers.' The keepers learn individually when to hold the ball and when to pass to a teammate, while the takers learn when to charge the ball-holder and when to cover possible passing lanes. Our agents learned policies that significantly out-performed a range of benchmark policies. We demonstrate the generality of our approach by applying it to a number of task variations including different field sizes and different numbers of players on each team.
Citation
P. Stone, R. Sutton. "Scaling Reinforcement Learning Toward RoboCup Soccer". International Conference on Machine Learning (ICML), Williams College, January 2001.| Keywords: | benchmark, linear, keepers, machine learning |
| Category: | In Conference |
BibTeX
@incollection{Stone+Sutton:ICML01,
author = {Peter Stone and Richard S. Sutton},
title = {Scaling Reinforcement Learning Toward RoboCup Soccer},
booktitle = {International Conference on Machine Learning (ICML)},
year = 2001,
}Last Updated: May 31, 2007Submitted by Staurt H. Johnson
