Not Logged In

Meta-learning for Predictive Knowledge Architectures: A Case Study Using TIDBD on a Sensor-rich Robotic Arm

Predictive approaches to modelling the environment have seen recent successes in robotics and other long-lived applications. These predictive knowledge architectures are learned incrementally and online, through interaction with the environment. One challenge for applications of predictive knowledge is the necessity of tuning feature representations and parameter values: no single step size will be appropriate for every prediction. Furthermore, as sensor signals might be subject to change in a non-stationary world, predefined step sizes cannot be sufficient for an autonomous agent. In this paper, we explore Temporal-Difference Incremental Delta-Bar-Delta (TIDBD)-a meta-learning method for temporal-difference (TD) learning which adapts a vector of many step sizes, allowing for simultaneous step size tuning and representation learning. We demonstrate that, for a predictive knowledge application, TIDBD is a viable alternative to tuning step-size parameters, by showing that the performance of TIDBD is comparable to that of TD with an exhaustive parameter search. Performance here is measured in terms of root mean squared difference from the true value, calculated offline. Moreover, TIDBD can perform representation learning, potentially supporting robust learning in the face of failing sensors. The ability for an autonomous agent to adapt its own learning and adjust its representation based on interactions with its environment is a key capability. With its potential to fulfill these desiderata, meta-learning is a promising component for future systems.

Citation

J. Günther, A. Kearney, N. Ady, M. Dawson, P. Pilarski. "Meta-learning for Predictive Knowledge Architectures: A Case Study Using TIDBD on a Sensor-rich Robotic Arm". Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), (ed: Edith Elkind, Manuela Veloso, Noa Agmon, Matthew E. Taylor), pp 1967–1969, May 2019.

Keywords: Computer systems organization, Embedded and cyber-physical systems, Robotics, Robotic autonomy, Hardware, Robustness, Computing methodologies, Machine learning, Learning paradigms, Reinforcement learning
Category: In Conference
Web Links: DOI
  ACM Digital Library

BibTeX

@incollection{Günther+al:AAMAS19,
  author = {Johannes Günther and Alex Kearney and Nadia M. Ady and Michael
    Rory Dawson and Patrick M. Pilarski},
  title = {Meta-learning for Predictive Knowledge Architectures: A Case Study
    Using TIDBD on a Sensor-rich Robotic Arm},
  Editor = {Edith Elkind, Manuela Veloso, Noa Agmon, Matthew E. Taylor},
  Pages = {1967–1969},
  booktitle = {Joint Conference on Autonomous Agents and Multi-Agent Systems
    (AAMAS)},
  year = 2019,
}

Last Updated: December 01, 2020
Submitted by Sabina P

University of Alberta Logo AICML Logo