The relativistic discriminator: a key element missing from standard GAN

The relativistic discriminator: a key element missing from standard GAN

10 Sep 2018 | Alexia Jolicoeur-Martineau
This paper introduces the concept of a "relativistic discriminator" as a key missing component in standard GANs (SGAN). The traditional discriminator in SGAN estimates the probability that input data is real, while the generator aims to make fake data appear real. However, the authors argue that the discriminator should also decrease the probability that real data is real, as this aligns with prior knowledge that half of the data in a mini-batch is fake. This adjustment is achieved through a relativistic discriminator, which estimates the probability that real data is more realistic than randomly sampled fake data. Two variants of this approach are presented: Relativistic GANs (RGANs) and Relativistic average GANs (RaGANs). These models are shown to be more stable and generate higher quality data samples compared to their non-relativistic counterparts. Empirical results demonstrate that RaGANs, when combined with gradient penalty, produce better quality images than WGAN-GP and SGAN with spectral normalization, even with fewer discriminator updates per generator update. Additionally, RaGANs are able to generate high-resolution images (256x256) from a small sample size (N=2011), outperforming other GAN variants in terms of quality. The paper also discusses the theoretical underpinnings of the relativistic discriminator, showing that it aligns with divergence minimization and provides a more stable training process. Overall, the relativistic discriminator improves the stability and quality of generated data in GANs.This paper introduces the concept of a "relativistic discriminator" as a key missing component in standard GANs (SGAN). The traditional discriminator in SGAN estimates the probability that input data is real, while the generator aims to make fake data appear real. However, the authors argue that the discriminator should also decrease the probability that real data is real, as this aligns with prior knowledge that half of the data in a mini-batch is fake. This adjustment is achieved through a relativistic discriminator, which estimates the probability that real data is more realistic than randomly sampled fake data. Two variants of this approach are presented: Relativistic GANs (RGANs) and Relativistic average GANs (RaGANs). These models are shown to be more stable and generate higher quality data samples compared to their non-relativistic counterparts. Empirical results demonstrate that RaGANs, when combined with gradient penalty, produce better quality images than WGAN-GP and SGAN with spectral normalization, even with fewer discriminator updates per generator update. Additionally, RaGANs are able to generate high-resolution images (256x256) from a small sample size (N=2011), outperforming other GAN variants in terms of quality. The paper also discusses the theoretical underpinnings of the relativistic discriminator, showing that it aligns with divergence minimization and provides a more stable training process. Overall, the relativistic discriminator improves the stability and quality of generated data in GANs.
Reach us at info@study.space