December 9, 2003 | Jennifer G. Dy, Carla E. Brodley
This paper addresses two key issues in developing an automated feature subset selection algorithm for unsupervised learning: determining the number of clusters in conjunction with feature selection and normalizing the bias of feature selection criteria with respect to dimension. The authors explore these issues through FSSEM (Feature Subset Selection using Expectation-Maximization (EM) clustering) and two performance criteria: scatter separability and maximum likelihood. They present proofs on the dimensionality biases of these criteria and propose a cross-projection normalization scheme to mitigate these biases. Their experiments show the importance of feature selection, the need to address these two issues, and the effectiveness of their proposed solutions.
The paper introduces FSSEM, a wrapper framework for unsupervised learning that incorporates the clustering algorithm into the feature search and selection process. It explores the scatter separability criterion, which measures cluster separation, and the maximum likelihood (ML) criterion, which evaluates how well a Gaussian mixture model fits the data. The authors demonstrate that both criteria have biases with respect to dimensionality, and propose a normalization scheme to address these biases.
The paper also discusses the challenge of determining the number of clusters in conjunction with feature selection, and presents FSSEM-k as a solution. The authors evaluate the performance of FSSEM using synthetic and real-world data, comparing the performance of different feature selection criteria. They find that the trace criterion outperforms the ML criterion in terms of cross-validated error, and that feature selection with k (FSSEM-k) is more effective than feature selection with a fixed k (FSSEM). The paper concludes with a discussion of the lessons learned and suggestions for future research.This paper addresses two key issues in developing an automated feature subset selection algorithm for unsupervised learning: determining the number of clusters in conjunction with feature selection and normalizing the bias of feature selection criteria with respect to dimension. The authors explore these issues through FSSEM (Feature Subset Selection using Expectation-Maximization (EM) clustering) and two performance criteria: scatter separability and maximum likelihood. They present proofs on the dimensionality biases of these criteria and propose a cross-projection normalization scheme to mitigate these biases. Their experiments show the importance of feature selection, the need to address these two issues, and the effectiveness of their proposed solutions.
The paper introduces FSSEM, a wrapper framework for unsupervised learning that incorporates the clustering algorithm into the feature search and selection process. It explores the scatter separability criterion, which measures cluster separation, and the maximum likelihood (ML) criterion, which evaluates how well a Gaussian mixture model fits the data. The authors demonstrate that both criteria have biases with respect to dimensionality, and propose a normalization scheme to address these biases.
The paper also discusses the challenge of determining the number of clusters in conjunction with feature selection, and presents FSSEM-k as a solution. The authors evaluate the performance of FSSEM using synthetic and real-world data, comparing the performance of different feature selection criteria. They find that the trace criterion outperforms the ML criterion in terms of cross-validated error, and that feature selection with k (FSSEM-k) is more effective than feature selection with a fixed k (FSSEM). The paper concludes with a discussion of the lessons learned and suggestions for future research.