Not Logged In

Correcting Covariate Shift with Frank-Wolfe Algorithm

Covariate shift is a fundamental problem for learning in non-stationary environments where the conditional distribution ppy|xq is the same between training and test data while their marginal distributions ptrpxq and ptepxq are different. Although many covariate shift correction techniques remain effective for real world problems, most do not scale well in practice. In this paper, using inspiration from recent optimization techniques, we apply the Frank-Wolfe algorithm to two well-known covariate shift correction techniques, Kernel Mean Matching (KMM) and Kullback-Leibler Importance Estimation Procedure (KLIEP), and identify an important connection between kernel herding and KMM. Our complexity analysis shows the benefits of the Frank-Wolfe approach over projected gradient methods in solving KMM and KLIEP. An empirical study then demonstrates the effectiveness and efficiency of the Frank-Wolfe algorithm for correcting covariate shift in practice.

Citation

J. Wen, R. Greiner, D. Schuurmans. "Correcting Covariate Shift with Frank-Wolfe Algorithm". International Joint Conference on Artificial Intelligence (IJCAI), (ed: Qiang Yang, Michael Wooldridge), pp 1010-1016, July 2015.

Keywords: machine learning, covariate shift, frank-wolfe
Category: In Conference
Web Links: IJCAI page

BibTeX

@incollection{Wen+al:IJCAI15,
  author = {Junfeng Wen and Russ Greiner and Dale Schuurmans},
  title = {Correcting Covariate Shift with Frank-Wolfe Algorithm},
  Editor = {Qiang Yang, Michael Wooldridge},
  Pages = {1010-1016},
  booktitle = {International Joint Conference on Artificial Intelligence
    (IJCAI)},
  year = 2015,
}

Last Updated: February 11, 2020
Submitted by Sabina P

University of Alberta Logo AICML Logo