Learning Representations for Automatic Colorization

Learning Representations for Automatic Colorization

13 Aug 2017 | Gustav Larsson1, Michael Maire2, and Gregory Shakhnarovich2
This paper presents a fully automatic image colorization system that leverages deep neural networks to predict per-pixel color histograms, enabling the generation of color images without human input. The system uses semantic information and deep convolutional networks to interpret scene content and localize objects, allowing for accurate color assignment. It outperforms existing methods on both fully and partially automatic colorization tasks, achieving state-of-the-art performance across multiple metrics and datasets. The system also explores colorization as a method for self-supervised visual representation learning, proposing a new ImageNet colorization benchmark. The approach involves training a deep convolutional network to predict hue and chroma distributions for each pixel based on spatially localized features. The network is trained end-to-end, with the output serving as an intermediate representation for color prediction. The system uses a hypercolumn architecture to combine features across multiple abstraction levels, enabling the prediction of color histograms rather than single colors. This allows for more accurate and flexible color assignments, particularly in complex images with ambiguous color information. The system is evaluated on two settings: fully automatic colorization (grayscale input only) and partially automatic colorization (grayscale input with reference global color histograms). It achieves the best performance across all metrics and datasets, outperforming prior methods that rely on additional information such as reference images or ground-truth color histograms. The system also demonstrates the effectiveness of colorization as a tool for learning visual representations, with results showing that it can be used to train models without relying on pre-trained ImageNet networks. The paper also presents additional results, including comparisons with other methods, and discusses the limitations of the system, such as challenges in handling ambiguous color information and the need for further refinement in certain scenarios. Overall, the system provides a novel and effective approach to automatic image colorization, with applications in graphics, historical image restoration, and visual learning.This paper presents a fully automatic image colorization system that leverages deep neural networks to predict per-pixel color histograms, enabling the generation of color images without human input. The system uses semantic information and deep convolutional networks to interpret scene content and localize objects, allowing for accurate color assignment. It outperforms existing methods on both fully and partially automatic colorization tasks, achieving state-of-the-art performance across multiple metrics and datasets. The system also explores colorization as a method for self-supervised visual representation learning, proposing a new ImageNet colorization benchmark. The approach involves training a deep convolutional network to predict hue and chroma distributions for each pixel based on spatially localized features. The network is trained end-to-end, with the output serving as an intermediate representation for color prediction. The system uses a hypercolumn architecture to combine features across multiple abstraction levels, enabling the prediction of color histograms rather than single colors. This allows for more accurate and flexible color assignments, particularly in complex images with ambiguous color information. The system is evaluated on two settings: fully automatic colorization (grayscale input only) and partially automatic colorization (grayscale input with reference global color histograms). It achieves the best performance across all metrics and datasets, outperforming prior methods that rely on additional information such as reference images or ground-truth color histograms. The system also demonstrates the effectiveness of colorization as a tool for learning visual representations, with results showing that it can be used to train models without relying on pre-trained ImageNet networks. The paper also presents additional results, including comparisons with other methods, and discusses the limitations of the system, such as challenges in handling ambiguous color information and the need for further refinement in certain scenarios. Overall, the system provides a novel and effective approach to automatic image colorization, with applications in graphics, historical image restoration, and visual learning.
Reach us at info@study.space