Improved Training of Wasserstein GANs

Improved Training of Wasserstein GANs

25 Dec 2017 | Ishaan Gulrajani1; Faruk Ahmed1, Martin Arjovsky2, Vincent Dumoulin1, Aaron Courville1,3
This paper introduces an improved method for training Wasserstein GANs (WGANs) by replacing the weight clipping technique with a gradient penalty. The authors find that weight clipping in WGANs can lead to unstable training and poor sample quality. Instead, they propose a gradient penalty that enforces the Lipschitz constraint on the critic by penalizing the norm of the gradient of the critic with respect to its input. This method performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with minimal hyperparameter tuning, including 101-layer ResNets and language models with continuous generators. The method also achieves high-quality image generation on CIFAR-10 and LSUN bedrooms. The paper demonstrates that the gradient penalty approach leads to more stable training, better sample quality, and improved performance over weight clipping. The authors also show that the gradient penalty method can be used to train complex architectures, including deep residual networks, and that it outperforms other GAN methods in terms of sample quality and training stability. The paper concludes that the gradient penalty approach is a more effective and stable method for training WGANs.This paper introduces an improved method for training Wasserstein GANs (WGANs) by replacing the weight clipping technique with a gradient penalty. The authors find that weight clipping in WGANs can lead to unstable training and poor sample quality. Instead, they propose a gradient penalty that enforces the Lipschitz constraint on the critic by penalizing the norm of the gradient of the critic with respect to its input. This method performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with minimal hyperparameter tuning, including 101-layer ResNets and language models with continuous generators. The method also achieves high-quality image generation on CIFAR-10 and LSUN bedrooms. The paper demonstrates that the gradient penalty approach leads to more stable training, better sample quality, and improved performance over weight clipping. The authors also show that the gradient penalty method can be used to train complex architectures, including deep residual networks, and that it outperforms other GAN methods in terms of sample quality and training stability. The paper concludes that the gradient penalty approach is a more effective and stable method for training WGANs.
Reach us at info@study.space