Are GANs Created Equal? A Large-Scale Study

Are GANs Created Equal? A Large-Scale Study

29 Oct 2018 | Mario Lucic*, Karol Kurach*, Marcin Michalski, Olivier Bousquet, Sylvain Gelly
This paper presents a large-scale empirical study on generative adversarial networks (GANs), aiming to assess which GAN algorithms perform best. The study evaluates state-of-the-art GANs using metrics such as Fréchet Inception Distance (FID) and precision/recall. The results show that most GANs can achieve similar FID scores with sufficient computational resources, suggesting that improvements often come from increased computational budget rather than fundamental algorithmic changes. The study also introduces new tasks where precision and recall can be measured, and highlights the importance of reporting distribution of results rather than just the best result. The authors find no evidence that any tested GAN consistently outperforms the non-saturating GAN. The study emphasizes the need for more systematic and objective evaluation procedures in GAN research. The paper also discusses the limitations of current metrics and proposes a more robust evaluation framework. The results indicate that the performance of GANs is highly sensitive to hyperparameters and that a wide range of hyperparameter settings is necessary to achieve good results. The study concludes that future GAN research should focus on more systematic and objective evaluation methods.This paper presents a large-scale empirical study on generative adversarial networks (GANs), aiming to assess which GAN algorithms perform best. The study evaluates state-of-the-art GANs using metrics such as Fréchet Inception Distance (FID) and precision/recall. The results show that most GANs can achieve similar FID scores with sufficient computational resources, suggesting that improvements often come from increased computational budget rather than fundamental algorithmic changes. The study also introduces new tasks where precision and recall can be measured, and highlights the importance of reporting distribution of results rather than just the best result. The authors find no evidence that any tested GAN consistently outperforms the non-saturating GAN. The study emphasizes the need for more systematic and objective evaluation procedures in GAN research. The paper also discusses the limitations of current metrics and proposes a more robust evaluation framework. The results indicate that the performance of GANs is highly sensitive to hyperparameters and that a wide range of hyperparameter settings is necessary to achieve good results. The study concludes that future GAN research should focus on more systematic and objective evaluation methods.
Reach us at info@study.space