Quantifying the Uncertainty of a Belief Net Response: Bayesian Error-Bars for Belief Net Inference
A Bayesian belief network models a joint distribution over variables using a
DAG to represent variable dependencies and network parameters to represent the
conditional probability of each variable given an assignment to its immediate
parents. Existing algorithms assume each network parameter is fixed. From a
Bayesian perspective, however, these network parameters can be random
variables that reflect uncertainty in parameter estimates, arising because the
parameters are learned from data, or as they are elicited from uncertain
experts.
Belief networks are commonly used to compute responses to queries --- ie,
return a number for
P(H=h|E=e). Parameter uncertainty induces uncertainty in
query responses, which are thus themselves random variables. This paper
investigates this query response distribution, and shows how to accurately
model it for any query and any network structure. In particular, we prove
that the query response is asymptotically Gaussian and provide its mean value
and asymptotic variance. Moreover, we present an algorithm for computing
these quantities that has the same worst-case complexity as inference in
general, and also describe straight-line code when the query includes all
n
variables. We provide empirical evidence that (1) our approximation of the
variance is very accurate, and (2) a Beta distribution with these moments
provides a very accurate model of the observed query response distribution.
We also show how to use this to produce accurate error bars around these
responses --- ie, to determine that the response to
P(H=h|E=e) is
x ± y
with confidence
1- δ.
Citation
T. Van Allen,
A. Singh,
R. Greiner,
P. Hooper.
"Quantifying the Uncertainty of a Belief Net Response: Bayesian Error-Bars for Belief Net Inference".
Artificial Intelligence (AIJ), September 2007.
| Keywords: |
Bayesian belief network, variance, bucket elimination, credible interval, error bars, machine learning |
| Category: |
In Journal |
| Web Links: |
Tech Note pages |
BibTeX
@article{VanAllen+al:AIJ07,
author = {Tim Van Allen and Ajit Singh and Russ Greiner and Peter Hooper},
title = {Quantifying the Uncertainty of a Belief Net Response: Bayesian
Error-Bars for Belief Net Inference},
journal = {Artificial Intelligence (AIJ)},
year = 2007,
}Last Updated: September 05, 2007
Submitted by Russ Greiner 
