31 Aug 2022 | Andreas Lugmayr Martin Danelljan Andres Romero Fisher Yu Radu Timofte Luc Van Gool
RePaint is a novel inpainting method that leverages Denoising Diffusion Probabilistic Models (DDPMs) to generate high-quality and semantically meaningful images. The method is designed to handle extreme masks and generalizes well to unseen mask types, addressing the limitations of existing approaches that often produce textural extensions rather than semantically correct content. By using a pre-trained unconditional DDPM as the generative prior, RePaint conditionally samples from the unmasked regions during the reverse diffusion process, allowing it to produce diverse and harmonized outputs. The method introduces an improved denoising strategy through resampling iterations to better condition the image, enhancing the quality and semantic accuracy of the generated images. Extensive experiments on CelebA-HQ and ImageNet datasets demonstrate that RePaint outperforms state-of-the-art autoregressive and GAN-based methods in terms of both quantitative metrics and user evaluations, achieving higher LPIPS scores and more realistic results.RePaint is a novel inpainting method that leverages Denoising Diffusion Probabilistic Models (DDPMs) to generate high-quality and semantically meaningful images. The method is designed to handle extreme masks and generalizes well to unseen mask types, addressing the limitations of existing approaches that often produce textural extensions rather than semantically correct content. By using a pre-trained unconditional DDPM as the generative prior, RePaint conditionally samples from the unmasked regions during the reverse diffusion process, allowing it to produce diverse and harmonized outputs. The method introduces an improved denoising strategy through resampling iterations to better condition the image, enhancing the quality and semantic accuracy of the generated images. Extensive experiments on CelebA-HQ and ImageNet datasets demonstrate that RePaint outperforms state-of-the-art autoregressive and GAN-based methods in terms of both quantitative metrics and user evaluations, achieving higher LPIPS scores and more realistic results.