Fast and Robust Fixed-Point Algorithms for Independent Component Analysis

Fast and Robust Fixed-Point Algorithms for Independent Component Analysis

MAY 1999 | Aapo Hyvärinen
This paper presents a novel approach to Independent Component Analysis (ICA) by combining information-theoretic and projection pursuit methods. The authors introduce a family of new contrast functions for ICA, which are based on maximum entropy approximations of differential entropy. These functions enable the estimation of the entire decomposition by minimizing mutual information and the estimation of individual independent components as projection pursuit directions. The statistical properties of the estimators are analyzed under the linear mixture model, and guidelines are provided for choosing robust and optimal contrast functions. Additionally, the paper introduces simple fixed-point algorithms for practical optimization of the contrast functions, which are shown to converge very fast and reliably. The effectiveness of the proposed methods is demonstrated through simulations and real-life experiments, including applications in blind source separation, feature extraction, and exploratory data analysis.This paper presents a novel approach to Independent Component Analysis (ICA) by combining information-theoretic and projection pursuit methods. The authors introduce a family of new contrast functions for ICA, which are based on maximum entropy approximations of differential entropy. These functions enable the estimation of the entire decomposition by minimizing mutual information and the estimation of individual independent components as projection pursuit directions. The statistical properties of the estimators are analyzed under the linear mixture model, and guidelines are provided for choosing robust and optimal contrast functions. Additionally, the paper introduces simple fixed-point algorithms for practical optimization of the contrast functions, which are shown to converge very fast and reliably. The effectiveness of the proposed methods is demonstrated through simulations and real-life experiments, including applications in blind source separation, feature extraction, and exploratory data analysis.
Reach us at info@study.space
[slides and audio] Fast and robust fixed-point algorithms for independent component analysis