Convex Two-Layer Modeling
Full Text: 4867-convex-two-layer-modeling.pdf
Latent variable prediction models, such as multi-layer networks, impose auxiliary latent variables between inputs and outputs to allow automatic inference of implicit features useful for prediction. Unfortunately, such models are difficult to train because inference over latent variables must be performed concurrently with parameter optimization---creating a highly non-convex problem. Instead of proposing another local training method, we develop a convex relaxation of hidden-layer conditional models that admits global training. Our approach extends current convex modeling approaches to handle two nested nonlinearities separated by a non-trivial adaptive latent layer. The resulting methods are able to acquire two-layer models that cannot be represented by any single-layer model over the same features, while improving training quality over local heuristics.
Citation
Ã. Aslan, H. Cheng, X. Zhang, D. Schuurmans. "Convex Two-Layer Modeling". Neural Information Processing Systems (NIPS), (ed: C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahraman, K. Q. Weinberger), pp 2985-2993, December 2013.| Keywords: | |
| Category: | In Conference |
| Web Links: | NeurIPS |
BibTeX
@incollection{Aslan+al:NIPS13,
author = {Özlem Aslan and Hao Cheng and Xinhua Zhang and Dale Schuurmans},
title = {Convex Two-Layer Modeling},
Editor = {C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahraman, K. Q.
Weinberger},
Pages = {2985-2993},
booktitle = {Neural Information Processing Systems (NIPS)},
year = 2013,
}Last Updated: February 19, 2020Submitted by Sabina P
