6 Dec 2017 | Martin Arjovsky, Soumith Chintala, and Léon Bottou
Wasserstein GAN (WGAN) is a generative adversarial network that addresses the instability and mode collapse issues of traditional GANs by using the Wasserstein distance (also known as Earth Mover's distance) as the loss function. Unlike traditional GANs, which use the Jensen-Shannon divergence or KL divergence, WGAN uses the Wasserstein distance, which is a more meaningful measure of the difference between two distributions. This distance is continuous and differentiable almost everywhere, making it more suitable for training generative models.
The key idea of WGAN is to train a discriminator (or critic) to estimate the Wasserstein distance between the real data distribution and the generated data distribution. The critic is trained to maximize the difference between the expected values of the discriminator on real and generated data. To ensure the critic is Lipschitz continuous, the weights of the critic are clipped to a small range. This approach allows for more stable training and avoids the mode collapse problem, where the generator produces samples that are limited to a small subset of the data distribution.
WGAN has shown significant improvements over traditional GANs in terms of training stability and sample quality. The Wasserstein distance provides a more meaningful loss metric that correlates with the generator's convergence and sample quality. Additionally, WGANs do not require careful balancing of the generator and discriminator training, and they can produce samples that are more diverse and realistic.
The paper presents a theoretical analysis of the Wasserstein distance compared to other probability distances and divergences. It shows that the Wasserstein distance is a more sensible cost function for learning distributions supported by low-dimensional manifolds. The paper also provides empirical results demonstrating the effectiveness of WGANs in image generation tasks, showing that they produce higher quality samples and are more stable during training. The Wasserstein distance allows for continuous and differentiable loss functions, which are essential for effective training of generative models.Wasserstein GAN (WGAN) is a generative adversarial network that addresses the instability and mode collapse issues of traditional GANs by using the Wasserstein distance (also known as Earth Mover's distance) as the loss function. Unlike traditional GANs, which use the Jensen-Shannon divergence or KL divergence, WGAN uses the Wasserstein distance, which is a more meaningful measure of the difference between two distributions. This distance is continuous and differentiable almost everywhere, making it more suitable for training generative models.
The key idea of WGAN is to train a discriminator (or critic) to estimate the Wasserstein distance between the real data distribution and the generated data distribution. The critic is trained to maximize the difference between the expected values of the discriminator on real and generated data. To ensure the critic is Lipschitz continuous, the weights of the critic are clipped to a small range. This approach allows for more stable training and avoids the mode collapse problem, where the generator produces samples that are limited to a small subset of the data distribution.
WGAN has shown significant improvements over traditional GANs in terms of training stability and sample quality. The Wasserstein distance provides a more meaningful loss metric that correlates with the generator's convergence and sample quality. Additionally, WGANs do not require careful balancing of the generator and discriminator training, and they can produce samples that are more diverse and realistic.
The paper presents a theoretical analysis of the Wasserstein distance compared to other probability distances and divergences. It shows that the Wasserstein distance is a more sensible cost function for learning distributions supported by low-dimensional manifolds. The paper also provides empirical results demonstrating the effectiveness of WGANs in image generation tasks, showing that they produce higher quality samples and are more stable during training. The Wasserstein distance allows for continuous and differentiable loss functions, which are essential for effective training of generative models.