Ladder Variational Autoencoders

Ladder Variational Autoencoders

27 May 2016 | Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, Ole Winther
This paper introduces the Ladder Variational Autoencoder (LVAE), a new inference model for Variational Autoencoders (VAEs) that improves generative performance by recursively correcting the generative distribution with a data-dependent approximate likelihood. The LVAE uses a top-down dependency structure in both inference and generative models, allowing it to combine bottom-up approximate likelihood information with top-down prior information from the generative distribution. This results in a tighter lower bound on the true log-likelihood and a deeper, more distributed hierarchy of latent variables compared to traditional VAEs. The paper shows that the LVAE outperforms VAEs and other generative models in terms of predictive log-likelihood and provides a more accurate latent representation of the data. It also demonstrates that batch normalization and deterministic warm-up are crucial for training deep stochastic models. The LVAE is shown to achieve better performance on benchmark datasets such as MNIST, OMNIGLOT, and NORB, with significant improvements in log-likelihood and latent representation quality. The paper also highlights the importance of hierarchical latent representations and the benefits of combining bottom-up and top-down information in the inference process. The results indicate that the LVAE provides a more structured and meaningful latent representation of the data, which is likely useful for semi-supervised learning.This paper introduces the Ladder Variational Autoencoder (LVAE), a new inference model for Variational Autoencoders (VAEs) that improves generative performance by recursively correcting the generative distribution with a data-dependent approximate likelihood. The LVAE uses a top-down dependency structure in both inference and generative models, allowing it to combine bottom-up approximate likelihood information with top-down prior information from the generative distribution. This results in a tighter lower bound on the true log-likelihood and a deeper, more distributed hierarchy of latent variables compared to traditional VAEs. The paper shows that the LVAE outperforms VAEs and other generative models in terms of predictive log-likelihood and provides a more accurate latent representation of the data. It also demonstrates that batch normalization and deterministic warm-up are crucial for training deep stochastic models. The LVAE is shown to achieve better performance on benchmark datasets such as MNIST, OMNIGLOT, and NORB, with significant improvements in log-likelihood and latent representation quality. The paper also highlights the importance of hierarchical latent representations and the benefits of combining bottom-up and top-down information in the inference process. The results indicate that the LVAE provides a more structured and meaningful latent representation of the data, which is likely useful for semi-supervised learning.
Reach us at info@study.space
[slides] Ladder Variational Autoencoders | StudySpace