Boltzmann Machine Learning With the Latent Maximum Entropy Principle
- Shaojun Wang, Dept of Computing Science
- Dale Schuurmans, AICML
- Fuchun Peng, Department of Computer Science, University of Massachusetts at Amherst
- Yunxin Zhao, Department of Computer Engineering and Computer Science, University of Missouri at Columbia
We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have pro- posed: the latent maximum entropy principle (LME). LME is dierent both from Jaynes' maximum entropy principle and from stan- dard maximum likelihood estimation. We demonstrate the LME principle by deriving new algorithms for Boltzmann machine pa- rameter estimation, and show how a robust and rapidly convergent new variant of the EM algorithm can be developed. Our exper- iments show that estimation based on LME generally yields better results than maximum likelihood estimation when inferring models from small amounts of data.
Citation
S. Wang, D. Schuurmans, F. Peng, Y. Zhao. "Boltzmann Machine Learning With the Latent Maximum Entropy Principle". Conference on Uncertainty in Artificial Intelligence (UAI), Acapulco, Mexico, August 2003.Keywords: | entropy, machine learning |
Category: | In Conference |
BibTeX
@incollection{Wang+al:UAI03, author = {Shaojun Wang and Dale Schuurmans and Fuchun Peng and Yunxin Zhao}, title = {Boltzmann Machine Learning With the Latent Maximum Entropy Principle}, booktitle = {Conference on Uncertainty in Artificial Intelligence (UAI)}, year = 2003, }Last Updated: June 01, 2007
Submitted by Staurt H. Johnson