| Jason Davis, Brian Kulis, Suvrit Sra and Inderjit Dhillon
This paper presents a novel approach to metric learning by formulating the problem in an information-theoretic framework. The goal is to learn a Mahalanobis distance matrix that satisfies given pairwise distance constraints. The authors show that this problem can be transformed into a low-rank kernel learning problem, which allows for efficient computation without requiring eigenvector calculations.
The key idea is to minimize the Burg divergence between a low-rank kernel and an input kernel, subject to pairwise distance constraints. This formulation provides a natural information-theoretic interpretation of the problem and offers insights into the relationship between metric learning and kernel learning. The algorithm leverages existing methods for low-rank kernel learning, which are faster than many existing metric learning techniques.
The problem is formulated as minimizing the Kullback-Leibler divergence between two multivariate Gaussians, subject to constraints on the Mahalanobis distance. The authors show that this problem is equivalent to a low-rank kernel learning problem, which can be solved efficiently using iterative optimization techniques. The algorithm avoids costly eigendecompositions and has a time complexity of O(cd²) per iteration, where c is the number of constraints and d is the dimensionality of the data.
The paper also discusses extensions of the basic framework, including the incorporation of slack variables to handle infeasible constraints and the adaptation of the method to different types of distance constraints. The authors compare their approach with other metric learning methods, highlighting the advantages of their information-theoretic formulation and efficient algorithm. The proposed method is shown to be more efficient and flexible, allowing for a wide range of applications in machine learning and data analysis.This paper presents a novel approach to metric learning by formulating the problem in an information-theoretic framework. The goal is to learn a Mahalanobis distance matrix that satisfies given pairwise distance constraints. The authors show that this problem can be transformed into a low-rank kernel learning problem, which allows for efficient computation without requiring eigenvector calculations.
The key idea is to minimize the Burg divergence between a low-rank kernel and an input kernel, subject to pairwise distance constraints. This formulation provides a natural information-theoretic interpretation of the problem and offers insights into the relationship between metric learning and kernel learning. The algorithm leverages existing methods for low-rank kernel learning, which are faster than many existing metric learning techniques.
The problem is formulated as minimizing the Kullback-Leibler divergence between two multivariate Gaussians, subject to constraints on the Mahalanobis distance. The authors show that this problem is equivalent to a low-rank kernel learning problem, which can be solved efficiently using iterative optimization techniques. The algorithm avoids costly eigendecompositions and has a time complexity of O(cd²) per iteration, where c is the number of constraints and d is the dimensionality of the data.
The paper also discusses extensions of the basic framework, including the incorporation of slack variables to handle infeasible constraints and the adaptation of the method to different types of distance constraints. The authors compare their approach with other metric learning methods, highlighting the advantages of their information-theoretic formulation and efficient algorithm. The proposed method is shown to be more efficient and flexible, allowing for a wide range of applications in machine learning and data analysis.