15 Dec 2018 | Guilin Liu Fitsum A. Reda Kevin J. Shih Ting-Chun Wang Andrew Tao Bryan Catanzaro
This paper introduces a novel approach to image inpainting using partial convolutions, which addresses the limitations of existing deep learning methods that often suffer from artifacts such as color discrepancies and blurriness. The proposed method uses a partial convolutional layer that is masked and renormalized to condition on only valid pixels, and includes an automatic mask update mechanism to handle irregular hole patterns. The model is evaluated on a large dataset of irregular masks and compared with state-of-the-art methods, demonstrating superior performance in terms of both qualitative and quantitative metrics. The authors also extend the framework to image super-resolution tasks, showing its effectiveness in these applications as well. The key contributions of the work include the introduction of partial convolutions and automatic mask updates, which enable the model to handle irregular holes robustly and without the need for post-processing.This paper introduces a novel approach to image inpainting using partial convolutions, which addresses the limitations of existing deep learning methods that often suffer from artifacts such as color discrepancies and blurriness. The proposed method uses a partial convolutional layer that is masked and renormalized to condition on only valid pixels, and includes an automatic mask update mechanism to handle irregular hole patterns. The model is evaluated on a large dataset of irregular masks and compared with state-of-the-art methods, demonstrating superior performance in terms of both qualitative and quantitative metrics. The authors also extend the framework to image super-resolution tasks, showing its effectiveness in these applications as well. The key contributions of the work include the introduction of partial convolutions and automatic mask updates, which enable the model to handle irregular holes robustly and without the need for post-processing.