This paper investigates the convergence properties of Generative Adversarial Networks (GANs) under different training methods. It demonstrates that the requirement of absolute continuity of distributions is necessary for local convergence, as unregularized GAN training can be non-convergent in more realistic scenarios where distributions are not absolutely continuous. The paper discusses various regularization strategies, including instance noise, zero-centered gradient penalties, and Wasserstein GANs (WGANs) and WGAN-GP, and shows that these methods affect convergence differently. Specifically, instance noise and zero-centered gradient penalties lead to local convergence, while WGANs and WGAN-GP do not always converge. The paper also introduces simplified gradient penalties and proves their local convergence under certain conditions. Finally, it provides experimental results showing that these penalties perform well in practice, enabling stable training of high-resolution generative models for various datasets with minimal hyperparameter tuning.This paper investigates the convergence properties of Generative Adversarial Networks (GANs) under different training methods. It demonstrates that the requirement of absolute continuity of distributions is necessary for local convergence, as unregularized GAN training can be non-convergent in more realistic scenarios where distributions are not absolutely continuous. The paper discusses various regularization strategies, including instance noise, zero-centered gradient penalties, and Wasserstein GANs (WGANs) and WGAN-GP, and shows that these methods affect convergence differently. Specifically, instance noise and zero-centered gradient penalties lead to local convergence, while WGANs and WGAN-GP do not always converge. The paper also introduces simplified gradient penalties and proves their local convergence under certain conditions. Finally, it provides experimental results showing that these penalties perform well in practice, enabling stable training of high-resolution generative models for various datasets with minimal hyperparameter tuning.