GANSpace: Discovering Interpretable GAN Controls

GANSpace: Discovering Interpretable GAN Controls

14 Dec 2020 | Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, Sylvain Paris
This paper introduces GANSpace, a method to discover interpretable controls for image synthesis in GANs. The approach uses Principal Component Analysis (PCA) to identify important latent directions in the latent or feature space, enabling the definition of interpretable controls such as viewpoint, aging, lighting, and time of day. The method allows layer-wise perturbation along these principal directions, and demonstrates that BigGAN can be controlled in a StyleGAN-like manner without retraining. Results are shown on various GANs trained on different datasets, demonstrating good qualitative matches to previously identified edit directions. The paper explores how PCA can be applied to find useful directions in the latent space of GANs. For StyleGAN, PCA is applied to the latent vectors, while for BigGAN, PCA is applied to intermediate layers and then transferred back to the latent space. These principal components are then used to create interpretable controls by applying them to specific layers. The method is simple algorithmically but leads to powerful controls, enabling manipulation of image attributes ranging from high-level properties like object pose and shape to more nuanced properties like lighting and facial attributes. The paper also discusses the properties of GAN principal components, showing that they can be used to identify and control various aspects of generated images. It highlights the entanglements between different concepts and the disallowed combinations that the model may not apply to certain faces. The method is compared to supervised methods and random directions, showing that PCA provides a useful ordering of directions, separating pose and appearance into the first components. The paper also presents a user interface that enables interactive exploration of the principal directions via simple slider controls, allowing layer-wise application of edits. The method is shown to be effective across various GAN models, including BigGAN and StyleGAN, and provides a way to discover biases and limitations in the input GANs. The approach does not require any training on images, instead taking an existing GAN as input and discovering techniques for controlling it. The method is simple but powerful, offering a way to analyze and control existing GANs without requiring expensive optimization or post-hoc supervision.This paper introduces GANSpace, a method to discover interpretable controls for image synthesis in GANs. The approach uses Principal Component Analysis (PCA) to identify important latent directions in the latent or feature space, enabling the definition of interpretable controls such as viewpoint, aging, lighting, and time of day. The method allows layer-wise perturbation along these principal directions, and demonstrates that BigGAN can be controlled in a StyleGAN-like manner without retraining. Results are shown on various GANs trained on different datasets, demonstrating good qualitative matches to previously identified edit directions. The paper explores how PCA can be applied to find useful directions in the latent space of GANs. For StyleGAN, PCA is applied to the latent vectors, while for BigGAN, PCA is applied to intermediate layers and then transferred back to the latent space. These principal components are then used to create interpretable controls by applying them to specific layers. The method is simple algorithmically but leads to powerful controls, enabling manipulation of image attributes ranging from high-level properties like object pose and shape to more nuanced properties like lighting and facial attributes. The paper also discusses the properties of GAN principal components, showing that they can be used to identify and control various aspects of generated images. It highlights the entanglements between different concepts and the disallowed combinations that the model may not apply to certain faces. The method is compared to supervised methods and random directions, showing that PCA provides a useful ordering of directions, separating pose and appearance into the first components. The paper also presents a user interface that enables interactive exploration of the principal directions via simple slider controls, allowing layer-wise application of edits. The method is shown to be effective across various GAN models, including BigGAN and StyleGAN, and provides a way to discover biases and limitations in the input GANs. The approach does not require any training on images, instead taking an existing GAN as input and discovering techniques for controlling it. The method is simple but powerful, offering a way to analyze and control existing GANs without requiring expensive optimization or post-hoc supervision.
Reach us at info@study.space