Not Logged In

Robust Learning under Uncertain Test Distributions

Many learning situations involve learning the conditional distribution p(y|x) when the training data is drawn from the training distribution p_{tr}(x), even though it will later be used to predict for instances drawn from a different test distribution p_{te}(x). Most current approaches focus on learning how to reweigh the training examples, to make them resemble the test distribution. However, reweighing does not always help, because (we show that) the test error also depends on the correctness of the underlying model class. This thesis analyses this situation by viewing the problem of learning under changing distributions as a game between a learner and an adversary. We characterize when such reweighing is needed, and also provide an algorithm, robust covariate shift adjustment (RCSA), that provides relevant weights. Our empirical studies, on UCI datasets and a real-world cancer prognostic prediction dataset, show that our analysis applies, and that our RCSA works eff ectively.

Citation

J. Wen. "Robust Learning under Uncertain Test Distributions". MSc Thesis, Dept of Computing Science, University of Alberta, September 2013.

Keywords: machine learning, robust, adversary, covariate shift
Category: MSc Thesis
Web Links: ERA

BibTeX

@mastersthesis{Wen:13,
  author = {Junfeng Wen},
  title = {Robust Learning under Uncertain Test Distributions},
  School = {Dept of Computing Science, University of Alberta},
  year = 2013,
}

Last Updated: December 24, 2013
Submitted by Russ Greiner

University of Alberta Logo AICML Logo