BAM: Bottleneck Attention Module

BAM: Bottleneck Attention Module

2018 | Jongchan Park, Sanghyun Woo, Joon-Young Lee, In So Kweon
Bottleneck Attention Module (BAM) is a simple and effective attention module that can be integrated with any feed-forward convolutional neural networks. The module infers an attention map along two separate pathways: channel and spatial. It is placed at each bottleneck of models where the downsampling of feature maps occurs. BAM constructs a hierarchical attention at bottlenecks with a number of parameters and is trainable in an end-to-end manner. The module is validated through extensive experiments on CIFAR-100, ImageNet-1K, VOC 2007, and MS COCO benchmarks, showing consistent improvement in classification and detection performances with various models. The code and models will be publicly available. The BAM module is designed to be lightweight and efficient, with a simple structure that decomposes the process of inferring a 3D attention map into two streams, significantly reducing computational and parametric overhead. The channel attention branch computes a channel attention map by aggregating feature maps across channels, while the spatial attention branch produces a spatial attention map by utilizing contextual information. The two attention maps are combined to produce a final 3D attention map, which is then used to refine the input feature map. Experiments on various tasks, including image classification and object detection, show that BAM outperforms existing baselines, demonstrating its wide applicability. The module is effective in improving performance with minimal parameter and computational overhead, making it suitable for use in compact networks and low-end devices. The results show that BAM can generalize well across different models and tasks, and its effectiveness is validated through extensive experiments on multiple benchmarks. The module's design is inspired by the human visual system, which has 'what' (channel) and 'where' (spatial) pathways, and both contribute to processing visual information. The BAM module is placed at the bottleneck of the network, which is the most critical point of information flow, and it improves the network's representation power by focusing on important elements. The module is effective in refining intermediate features and enhancing the network's capacity. The results show that BAM can significantly improve the performance of networks with a fewer number of parameters, and it is suitable for use in various applications, including image classification and object detection.Bottleneck Attention Module (BAM) is a simple and effective attention module that can be integrated with any feed-forward convolutional neural networks. The module infers an attention map along two separate pathways: channel and spatial. It is placed at each bottleneck of models where the downsampling of feature maps occurs. BAM constructs a hierarchical attention at bottlenecks with a number of parameters and is trainable in an end-to-end manner. The module is validated through extensive experiments on CIFAR-100, ImageNet-1K, VOC 2007, and MS COCO benchmarks, showing consistent improvement in classification and detection performances with various models. The code and models will be publicly available. The BAM module is designed to be lightweight and efficient, with a simple structure that decomposes the process of inferring a 3D attention map into two streams, significantly reducing computational and parametric overhead. The channel attention branch computes a channel attention map by aggregating feature maps across channels, while the spatial attention branch produces a spatial attention map by utilizing contextual information. The two attention maps are combined to produce a final 3D attention map, which is then used to refine the input feature map. Experiments on various tasks, including image classification and object detection, show that BAM outperforms existing baselines, demonstrating its wide applicability. The module is effective in improving performance with minimal parameter and computational overhead, making it suitable for use in compact networks and low-end devices. The results show that BAM can generalize well across different models and tasks, and its effectiveness is validated through extensive experiments on multiple benchmarks. The module's design is inspired by the human visual system, which has 'what' (channel) and 'where' (spatial) pathways, and both contribute to processing visual information. The BAM module is placed at the bottleneck of the network, which is the most critical point of information flow, and it improves the network's representation power by focusing on important elements. The module is effective in refining intermediate features and enhancing the network's capacity. The results show that BAM can significantly improve the performance of networks with a fewer number of parameters, and it is suitable for use in various applications, including image classification and object detection.
Reach us at info@study.space
[slides and audio] BAM%3A Bottleneck Attention Module