This paper proposes spectral normalization as a method to stabilize the training of generative adversarial networks (GANs). The key idea is to normalize the weight matrices of the discriminator to ensure that the Lipschitz constant is controlled, which helps prevent instability during training. The proposed method, called spectral normalization, is computationally efficient and easy to integrate into existing GAN implementations. The authors tested their method on CIFAR10, STL-10, and ILSVRC2012 datasets and found that spectrally normalized GANs (SN-GANs) generate images of better or equal quality compared to previous stabilization techniques.
The method works by normalizing the spectral norm of each layer's weight matrix so that the maximum singular value is 1. This ensures that the discriminator remains Lipschitz continuous, which is crucial for stable training. The authors also compared their method with other regularization techniques such as weight normalization, weight clipping, and gradient penalty. They found that spectral normalization outperformed these methods in terms of image quality and stability.
The authors also discussed the computational efficiency of their method, noting that it is much faster than other regularization techniques like WGAN-GP. They demonstrated that their method can be applied to large datasets and that it produces high-quality images with a good inception score and low Fréchet inception distance. The code for the method is available at https://github.com/pfnet-research/sngan_projection.This paper proposes spectral normalization as a method to stabilize the training of generative adversarial networks (GANs). The key idea is to normalize the weight matrices of the discriminator to ensure that the Lipschitz constant is controlled, which helps prevent instability during training. The proposed method, called spectral normalization, is computationally efficient and easy to integrate into existing GAN implementations. The authors tested their method on CIFAR10, STL-10, and ILSVRC2012 datasets and found that spectrally normalized GANs (SN-GANs) generate images of better or equal quality compared to previous stabilization techniques.
The method works by normalizing the spectral norm of each layer's weight matrix so that the maximum singular value is 1. This ensures that the discriminator remains Lipschitz continuous, which is crucial for stable training. The authors also compared their method with other regularization techniques such as weight normalization, weight clipping, and gradient penalty. They found that spectral normalization outperformed these methods in terms of image quality and stability.
The authors also discussed the computational efficiency of their method, noting that it is much faster than other regularization techniques like WGAN-GP. They demonstrated that their method can be applied to large datasets and that it produces high-quality images with a good inception score and low Fréchet inception distance. The code for the method is available at https://github.com/pfnet-research/sngan_projection.