Colorful Image Colorization

Colorful Image Colorization

5 Oct 2016 | Richard Zhang, Phillip Isola, Alexei A. Efros
This paper presents a fully automatic method for image colorization that produces vibrant and realistic color versions of grayscale photographs. The approach treats the colorization problem as a classification task, leveraging class-rebalancing during training to enhance color diversity. The system is implemented as a feed-forward CNN, trained on over a million color images. The method outperforms previous approaches in a "colorization Turing test," fooling humans on 32% of trials, significantly higher than prior methods. It also demonstrates that colorization can serve as a powerful pretext task for self-supervised feature learning, acting as a cross-channel encoder. The method achieves state-of-the-art performance on several feature learning benchmarks. The system predicts the a and b color channels of an image in the CIE Lab colorspace, given the lightness channel L. To address the multimodal nature of the problem, the model predicts a distribution of possible colors for each pixel and reweights the loss to emphasize rare colors. The final colorization is obtained by taking the annealed-mean of the distribution, resulting in more vibrant and perceptually realistic colorizations. The paper evaluates the quality of synthesized images using a novel "colorization Turing test," where participants are asked to distinguish between real and synthesized color images. The method successfully fools participants on 32% of trials, indicating nearly photorealistic results. The system's colorizations are also shown to be useful for downstream tasks, such as object classification, using an off-the-shelf VGG network. The paper also explores colorization as a form of self-supervised representation learning, where raw data is used as its own source of supervision. The method achieves state-of-the-art results on several benchmarks, demonstrating that colorization can be a competitive and effective method for self-supervised feature learning. The system is trained on a million color images and achieves strong performance on tasks such as image classification, detection, and segmentation on the PASCAL dataset. The paper concludes that image colorization is a challenging pixel prediction problem in computer vision, and that a deep CNN with a well-chosen objective function can produce results indistinguishable from real color photos. The method not only provides a useful graphics output but also serves as a pretext task for representation learning, achieving strong performance in object classification, detection, and segmentation.This paper presents a fully automatic method for image colorization that produces vibrant and realistic color versions of grayscale photographs. The approach treats the colorization problem as a classification task, leveraging class-rebalancing during training to enhance color diversity. The system is implemented as a feed-forward CNN, trained on over a million color images. The method outperforms previous approaches in a "colorization Turing test," fooling humans on 32% of trials, significantly higher than prior methods. It also demonstrates that colorization can serve as a powerful pretext task for self-supervised feature learning, acting as a cross-channel encoder. The method achieves state-of-the-art performance on several feature learning benchmarks. The system predicts the a and b color channels of an image in the CIE Lab colorspace, given the lightness channel L. To address the multimodal nature of the problem, the model predicts a distribution of possible colors for each pixel and reweights the loss to emphasize rare colors. The final colorization is obtained by taking the annealed-mean of the distribution, resulting in more vibrant and perceptually realistic colorizations. The paper evaluates the quality of synthesized images using a novel "colorization Turing test," where participants are asked to distinguish between real and synthesized color images. The method successfully fools participants on 32% of trials, indicating nearly photorealistic results. The system's colorizations are also shown to be useful for downstream tasks, such as object classification, using an off-the-shelf VGG network. The paper also explores colorization as a form of self-supervised representation learning, where raw data is used as its own source of supervision. The method achieves state-of-the-art results on several benchmarks, demonstrating that colorization can be a competitive and effective method for self-supervised feature learning. The system is trained on a million color images and achieves strong performance on tasks such as image classification, detection, and segmentation on the PASCAL dataset. The paper concludes that image colorization is a challenging pixel prediction problem in computer vision, and that a deep CNN with a well-chosen objective function can produce results indistinguishable from real color photos. The method not only provides a useful graphics output but also serves as a pretext task for representation learning, achieving strong performance in object classification, detection, and segmentation.
Reach us at info@study.space
[slides and audio] Colorful Image Colorization