Unsupervised Image-to-Image Translation Networks

Unsupervised Image-to-Image Translation Networks

23 Jul 2018 | Ming-Yu Liu, Thomas Breuel, Jan Kautz
The paper presents an unsupervised image-to-image translation framework based on Coupled GANs, which aims to learn the joint distribution of images in different domains using only marginal distributions. The key challenge is that there are infinitely many joint distributions that can produce the given marginal distributions without additional assumptions. To address this, the authors propose a shared-latent space assumption, where corresponding images in different domains are mapped to the same latent representation in a shared latent space. The framework consists of two VAE-GANs, each modeling one image domain, with adversarial training and weight-sharing constraints to enforce the shared-latent space assumption. The framework is evaluated on various challenging tasks, including street scene, animal, and face image translation, achieving high-quality results. Additionally, it is applied to domain adaptation tasks, outperforming state-of-the-art methods on benchmark datasets. The code and additional results are available at https://github.com/mingyulitwu/unit.The paper presents an unsupervised image-to-image translation framework based on Coupled GANs, which aims to learn the joint distribution of images in different domains using only marginal distributions. The key challenge is that there are infinitely many joint distributions that can produce the given marginal distributions without additional assumptions. To address this, the authors propose a shared-latent space assumption, where corresponding images in different domains are mapped to the same latent representation in a shared latent space. The framework consists of two VAE-GANs, each modeling one image domain, with adversarial training and weight-sharing constraints to enforce the shared-latent space assumption. The framework is evaluated on various challenging tasks, including street scene, animal, and face image translation, achieving high-quality results. Additionally, it is applied to domain adaptation tasks, outperforming state-of-the-art methods on benchmark datasets. The code and additional results are available at https://github.com/mingyulitwu/unit.
Reach us at info@study.space
Understanding Unsupervised Image-to-Image Translation Networks