View Publication

Semantic N-Gram Language Modelling With the Latent Maximum Entropy Principle

Shaojun Wang, Dept of Computing Science
Dale Schuurmans, AICML
Fuchun Peng, Department of Computer Science, University of Massachusetts at Amherst
Yunxin Zhao, Department of Computer Engineering and Computer Science, University of Missouri at Columbia

In this paper, we describe a unified probabilistic framework for statistical language modeling---the latent maximum en� tropy principle---which can e#ectively incorporate various aspects of natural language, such as local word interac� tion, syntactic structure and semantic document informa� tion. Unlike previous work on maximum entropy methods for language modeling, which only allow explicit features to be modeled, our framework also allows relationships over hidden features to be captured, resulting in a more ex� pressive language model. We describe e#cient algorithms for marginalization, inference and normalization in our ex� tended models. We then present experimental results for our approach on the Wall Street Journal corpus.

Citation

S. Wang, D. Schuurmans, F. Peng, Y. Zhao. "Semantic N-Gram Language Modelling With the Latent Maximum Entropy Principle". ICASSP, May 2003.

Keywords:	semantic, machine learning
Category:	In Conference

BibTeX

@incollection{Wang+al:ICASSP03,
  author = {Shaojun Wang and Dale Schuurmans and Fuchun Peng and Yunxin Zhao},
  title = {Semantic N-Gram Language Modelling With the Latent Maximum Entropy
    Principle},
  booktitle = {},
  year = 2003,
}

Last Updated: March 14, 2007
Submitted by AICML Admin Assistant

Not Logged In

PapersDB

Semantic N-Gram Language Modelling With the Latent Maximum Entropy Principle

Citation

BibTeX