TOWARDS PRINCIPLED METHODS FOR TRAINING GENERATIVE ADVERSARIAL NETWORKS

TOWARDS PRINCIPLED METHODS FOR TRAINING GENERATIVE ADVERSARIAL NETWORKS

17 Jan 2017 | Martin Arjovsky, Léon Bottou
This paper aims to provide a theoretical foundation for understanding the training dynamics of Generative Adversarial Networks (GANs). It is divided into three sections: an introduction, a detailed analysis of the problems (instability and saturation), and a practical approach to solving these issues. The authors introduce new tools to study GANs and address the instability and vanishing gradients problems. They prove that if the distributions being generated have disjoint supports or lie on low-dimensional manifolds, the optimal discriminator will be perfect and its gradient will be zero almost everywhere, leading to unstable training. To mitigate this, they propose adding continuous noise to the inputs of the discriminator, which smooths the distribution and provides interpretable gradients. The paper also introduces the Wasserstein metric as a softer measure of similarity between distributions, which can be used to evaluate generative models regardless of whether they are continuous or discrete. The authors conclude by stating several open questions and directions for future research.This paper aims to provide a theoretical foundation for understanding the training dynamics of Generative Adversarial Networks (GANs). It is divided into three sections: an introduction, a detailed analysis of the problems (instability and saturation), and a practical approach to solving these issues. The authors introduce new tools to study GANs and address the instability and vanishing gradients problems. They prove that if the distributions being generated have disjoint supports or lie on low-dimensional manifolds, the optimal discriminator will be perfect and its gradient will be zero almost everywhere, leading to unstable training. To mitigate this, they propose adding continuous noise to the inputs of the discriminator, which smooths the distribution and provides interpretable gradients. The paper also introduces the Wasserstein metric as a softer measure of similarity between distributions, which can be used to evaluate generative models regardless of whether they are continuous or discrete. The authors conclude by stating several open questions and directions for future research.
Reach us at info@study.space
Understanding Towards Principled Methods for Training Generative Adversarial Networks