Discriminative Model Selection for Belief Net Structures
- Yuhong Guo
- Russ Greiner, Dept of Computing Science; PI of AICML
Bayesian belief nets (BNs) are often used for classification tasks, typicallyto return the most likely class label for a specified instance. Many BN-learners, however, attempt to find the BN that maximizes a different objective function --- viz., likelihood, rather than classification accuracy--- typically by first using some model selection criterion to identify an appropriate graphical structure, then finding good parameters for thatstructure. This paper considers a number of possible criteria for selectingthe best structure, both generative (ie, based on likelihood; BIC, BDe) and discriminative (ie, Conditional BIC (CBIC), resubstitution ClassificationError (CE) and Bias2+Variance (BV) ). We empirically compare these criteria against a variety of different 'correct BN structures', both real-world and synthetic, over a range of complexities. We also explore different ways to set the parameters, dealing with two issues: (1) Should we seek the parameters that maximize likelihood versus the ones that maximize conditional likelihood? (2) Should we use (i) the entire training sample first to learn the best parameters and then to evaluate the models, versus (ii) only a partition for parameter estimation and another partition for evaluation (cross-validation)? Our results show that the discriminative BV model selection criterion is one of the best measures for identifying the optimal structure, while the discriminative CBIC performs poorly; that we should use the parameters that maximize likelihood; and that it is typically better to use cross-validation here.
Citation
Y. Guo, R. Greiner. "Discriminative Model Selection for Belief Net Structures". National Conference on Artificial Intelligence (AAAI), Pittsburgh, pp 770-776, July 2005.| Keywords: | bayesian classifier, belief nets, machine learning |
| Category: | In Conference |
BibTeX
@incollection{Guo+Greiner:AAAI05,
author = {Yuhong Guo and Russ Greiner},
title = {Discriminative Model Selection for Belief Net Structures},
Pages = {770-776},
booktitle = {National Conference on Artificial Intelligence (AAAI)},
year = 2005,
}Last Updated: April 23, 2007Submitted by Nelson Loyola
