Extracting and Composing Robust Features with Denoising Autoencoders

Extracting and Composing Robust Features with Denoising Autoencoders

February 2008 | Pascal Vincent, Hugo Larochelle, Yoshua Bengio, Pierre-Antoine Manzagol
The paper introduces a new training principle for unsupervised learning of representations using denoising autoencoders, which are trained to reconstruct clean inputs from corrupted ones. This approach aims to create robust representations that can handle partial corruption of the input pattern. The authors motivate this method from manifold learning, information theory, and generative model perspectives. They demonstrate that this technique significantly improves the performance of deep architectures in pattern classification tasks, outperforming other methods such as stacked autoencoders and deep belief networks. The experiments show that the denoising autoencoders learn more useful feature detectors, especially at higher noise levels, which capture larger structures across multiple input dimensions. The paper concludes by suggesting future work on exploring different types of corruption processes and their impact on representation learning.The paper introduces a new training principle for unsupervised learning of representations using denoising autoencoders, which are trained to reconstruct clean inputs from corrupted ones. This approach aims to create robust representations that can handle partial corruption of the input pattern. The authors motivate this method from manifold learning, information theory, and generative model perspectives. They demonstrate that this technique significantly improves the performance of deep architectures in pattern classification tasks, outperforming other methods such as stacked autoencoders and deep belief networks. The experiments show that the denoising autoencoders learn more useful feature detectors, especially at higher noise levels, which capture larger structures across multiple input dimensions. The paper concludes by suggesting future work on exploring different types of corruption processes and their impact on representation learning.
Reach us at info@study.space