13 Jul 2017 | Raymond A. Yeh*, Chen Chen*, Teck Yian Lim, Alexander G. Schwing, Mark Hasegawa-Johnson, Minh N. Do
This paper presents a novel method for semantic image inpainting, which aims to fill large missing regions in images based on available visual data. Traditional methods often fail due to the lack of high-level context, while learning-based methods require specific information about the holes during training. The proposed method leverages deep generative models, specifically an adversarial network (GAN), to generate missing content by conditioning on the available data. The process involves searching for the closest encoding of the corrupted image in the latent space using context and prior losses. This encoding is then used to infer the missing content through the generative model. The method is evaluated on three datasets (CelebA, SVHN, and Stanford Cars) and shows superior performance in predicting realistic images, outperforming state-of-the-art methods in terms of pixel-level photorealism. The key contributions include the use of a weighted context loss and a prior loss to ensure realistic and contextually appropriate results.This paper presents a novel method for semantic image inpainting, which aims to fill large missing regions in images based on available visual data. Traditional methods often fail due to the lack of high-level context, while learning-based methods require specific information about the holes during training. The proposed method leverages deep generative models, specifically an adversarial network (GAN), to generate missing content by conditioning on the available data. The process involves searching for the closest encoding of the corrupted image in the latent space using context and prior losses. This encoding is then used to infer the missing content through the generative model. The method is evaluated on three datasets (CelebA, SVHN, and Stanford Cars) and shows superior performance in predicting realistic images, outperforming state-of-the-art methods in terms of pixel-level photorealism. The key contributions include the use of a weighted context loss and a prior loss to ensure realistic and contextually appropriate results.