A Note on Metric Properties of Some Divergence Measures. The Gaussian Case
- Karim Tamer Abou-Moustafa, AICML and Dept. of Computing Science
- Frank Ferrie, McGill University
Multivariate Gaussian densities are pervasive in pattern recognition and machine learning. A central operation that appears in most of these areas is to measure the difference between two multivariate Gaussians. Unfortunately, traditional measures based on the Kullback--Leibler (KL) divergence and the Bhattacharyya distance do not satisfy all metric axioms necessary for many algorithms. In this paper we propose a modification for the KL divergence and the Bhattacharyya distance, for multivariate Gaussian densities, that transforms the two measures into distance metrics. Next, we show how these metric axioms impact the unfolding process of manifold learning algorithms. Finally, we illustrate the efficacy of the proposed metrics on two different manifold learning algorithms when used for motion clustering in video data. Our results show that, in this particular application, the new proposed metrics lead to boosts in performance (at least $7%$) when compared to other divergence measures.
Citation
K. Abou-Moustafa, F. Ferrie. "A Note on Metric Properties of Some Divergence Measures. The Gaussian Case". Asian Conference on Machine Learning, (ed: Steven C.H. Hoi and Wray Buntine), pp 1-15, November 2012.| Keywords: | Divergence measures, Gaussian densities, manifold learning, Riemannian metric for covariance matrices. |
| Category: | In Conference |
BibTeX
@incollection{Abou-Moustafa+Ferrie:ACML12,
author = {Karim Tamer Abou-Moustafa and Frank Ferrie},
title = {A Note on Metric Properties of Some Divergence Measures. The
Gaussian Case},
Editor = {Steven C.H. Hoi and Wray Buntine},
Pages = {1-15},
booktitle = {Asian Conference on Machine Learning},
year = 2012,
}Last Updated: September 29, 2013Submitted by Karim T. Abou-Moustafa
