Attention to Scale: Scale-aware Semantic Image Segmentation

Attention to Scale: Scale-aware Semantic Image Segmentation

2 Jun 2016 | Liang-Chieh Chen, Yi Yang, Jiang Wang, Wei Xu, Alan L. Yuille
This paper addresses the challenge of semantic image segmentation by proposing an attention mechanism that learns to weight multi-scale features at each pixel location. The authors adapt a state-of-the-art semantic segmentation model, DeepLab, and jointly train it with multi-scale input images and an attention model. The attention model, parameterized by another fully convolutional network, learns to assign weights to features from different scales, allowing for better fusion of multi-scale information. The proposed method outperforms average-max-pooling and provides diagnostic visualization of feature importance at different scales and positions. Extensive experiments on challenging datasets, including PASCAL-Person-Part, PASCAL VOC 2012, and a subset of MS-COCO 2014, demonstrate the effectiveness of the proposed model. The results show that the attention model improves performance and provides interpretable weight maps, while adding extra supervision to the output of each scale is essential for achieving excellent results.This paper addresses the challenge of semantic image segmentation by proposing an attention mechanism that learns to weight multi-scale features at each pixel location. The authors adapt a state-of-the-art semantic segmentation model, DeepLab, and jointly train it with multi-scale input images and an attention model. The attention model, parameterized by another fully convolutional network, learns to assign weights to features from different scales, allowing for better fusion of multi-scale information. The proposed method outperforms average-max-pooling and provides diagnostic visualization of feature importance at different scales and positions. Extensive experiments on challenging datasets, including PASCAL-Person-Part, PASCAL VOC 2012, and a subset of MS-COCO 2014, demonstrate the effectiveness of the proposed model. The results show that the attention model improves performance and provides interpretable weight maps, while adding extra supervision to the output of each scale is essential for achieving excellent results.
Reach us at info@study.space
[slides] Attention to Scale%3A Scale-Aware Semantic Image Segmentation | StudySpace