5 Oct 2016 | Richard Zhang, Phillip Isola, Alexei A. Efros
This paper addresses the problem of automatically colorizing grayscale photographs to produce plausible and vibrant color versions. The authors propose a fully automatic approach that uses a deep convolutional neural network (CNN) to predict color distributions for each pixel, leveraging large-scale training data. They address the underconstrained nature of the problem by posing it as a classification task and using class-rebalancing to increase the diversity of colors in the output. The system is trained on over a million color images and evaluated using a "colorization Turing test" where human participants are asked to distinguish between generated and ground truth color images. The method successfully fools humans on 32% of trials, significantly higher than previous methods. Additionally, the authors show that colorization can be used as a pretext task for self-supervised feature learning, achieving state-of-the-art performance on several benchmarks. The paper contributes to both the graphics problem of automatic image colorization and the field of self-supervised representation learning.This paper addresses the problem of automatically colorizing grayscale photographs to produce plausible and vibrant color versions. The authors propose a fully automatic approach that uses a deep convolutional neural network (CNN) to predict color distributions for each pixel, leveraging large-scale training data. They address the underconstrained nature of the problem by posing it as a classification task and using class-rebalancing to increase the diversity of colors in the output. The system is trained on over a million color images and evaluated using a "colorization Turing test" where human participants are asked to distinguish between generated and ground truth color images. The method successfully fools humans on 32% of trials, significantly higher than previous methods. Additionally, the authors show that colorization can be used as a pretext task for self-supervised feature learning, achieving state-of-the-art performance on several benchmarks. The paper contributes to both the graphics problem of automatic image colorization and the field of self-supervised representation learning.