7 Nov 2016 | Yaniv Taigman, Adam Polyak & Lior Wolf
This paper presents a method for unsupervised cross-domain image generation, where the goal is to transfer a sample from one domain to an analogous sample in another domain. The Domain Transfer Network (DTN) is proposed, which uses a compound loss function that includes a multiclass GAN loss, an f-constancy component, and a regularizing component that encourages G to map samples from T to themselves. The method is applied to visual domains, including digits and face images, and demonstrates the ability to generate convincing novel images of previously unseen entities while preserving their identity.
The DTN employs a deep neural network structure where the function G is a composition of the input function f and a learned function g. A compound loss function is used, which includes a GAN term that encourages the creation of samples that are indistinguishable from the training samples of the target domain, an f-constancy term that ensures f(x) is approximately equal to f(G(x)), and a regularizer that encourages G to be the identity mapping for all x in T.
The method is tested on two visual domains: digits and face images. In the digits domain, the method transfers images from the SVHN dataset to the MNIST dataset, achieving a test error of 4.95%. In the face domain, the method transfers a set of random and unlabeled face images to a space of emoji images, producing visually appealing results that capture more facial characteristics than those created by human annotators.
The method is also compared to other approaches, including style transfer and domain adaptation. It is shown that the DTN outperforms these methods in terms of f-constancy and domain transfer performance. The method is also able to generate emoji images that are more informative than those created by human annotators, and can be used for unsupervised domain adaptation. The results demonstrate that the DTN is a promising approach for cross-domain image generation and domain adaptation.This paper presents a method for unsupervised cross-domain image generation, where the goal is to transfer a sample from one domain to an analogous sample in another domain. The Domain Transfer Network (DTN) is proposed, which uses a compound loss function that includes a multiclass GAN loss, an f-constancy component, and a regularizing component that encourages G to map samples from T to themselves. The method is applied to visual domains, including digits and face images, and demonstrates the ability to generate convincing novel images of previously unseen entities while preserving their identity.
The DTN employs a deep neural network structure where the function G is a composition of the input function f and a learned function g. A compound loss function is used, which includes a GAN term that encourages the creation of samples that are indistinguishable from the training samples of the target domain, an f-constancy term that ensures f(x) is approximately equal to f(G(x)), and a regularizer that encourages G to be the identity mapping for all x in T.
The method is tested on two visual domains: digits and face images. In the digits domain, the method transfers images from the SVHN dataset to the MNIST dataset, achieving a test error of 4.95%. In the face domain, the method transfers a set of random and unlabeled face images to a space of emoji images, producing visually appealing results that capture more facial characteristics than those created by human annotators.
The method is also compared to other approaches, including style transfer and domain adaptation. It is shown that the DTN outperforms these methods in terms of f-constancy and domain transfer performance. The method is also able to generate emoji images that are more informative than those created by human annotators, and can be used for unsupervised domain adaptation. The results demonstrate that the DTN is a promising approach for cross-domain image generation and domain adaptation.