SFC: Shared Feature Calibration in Weakly Supervised Semantic Segmentation

SFC: Shared Feature Calibration in Weakly Supervised Semantic Segmentation

22 Jan 2024 | Xinqiao Zhao, Feilong Tang, Xiaoyang Wang, Jimin Xiao
This paper introduces a novel method called Shared Feature Calibration (SFC) for weakly supervised semantic segmentation. The main issue addressed is the over-activation of head classes and under-activation of tail classes in Class Activation Mapping (CAM) due to long-tailed data distributions. SFC aims to balance the shared feature proportions in classifier weights to improve CAM quality and segmentation performance. The SFC method involves two key components: Image Bank Re-sampling (IBR) and Multi-Scaled Distribution-Weighted (MSDW) consistency loss. IBR increases the sampling frequency of tail classes, while MSDW loss adjusts the consistency between CAMs generated by classifier weights and class prototypes. The MSDW loss is weighted by the sample number gaps between classes, ensuring that tail classes receive more attention during training. The proposed method improves CAM boundaries and achieves state-of-the-art results on Pascal VOC 2012 and COCO 2014 datasets. The SFC method enhances the performance of weakly supervised semantic segmentation by addressing the over-/under-activation issues caused by long-tailed data distributions. The final CAM is obtained by combining the classifier weight CAM and prototype CAM, which jointly solves the over-/under-activation problem. The paper also provides a detailed analysis of how shared features in classifier weights lead to over-/under-activation issues under long-tailed scenarios. Theoretical analysis shows that the shared feature component in head-class classifier weights tends to be positive, while that in tail-class classifier weights tends to be negative. This leads to over-activation of head classes and under-activation of tail classes in CAMs. The SFC method is evaluated on two benchmark datasets, Pascal VOC 2012 and COCO 2014, and achieves significant improvements in segmentation performance. The results show that SFC outperforms existing methods in terms of mIoU scores, demonstrating its effectiveness in improving weakly supervised semantic segmentation. The method is also validated through ablation studies, showing that the combination of IBR and MSDW loss is crucial for achieving the best performance.This paper introduces a novel method called Shared Feature Calibration (SFC) for weakly supervised semantic segmentation. The main issue addressed is the over-activation of head classes and under-activation of tail classes in Class Activation Mapping (CAM) due to long-tailed data distributions. SFC aims to balance the shared feature proportions in classifier weights to improve CAM quality and segmentation performance. The SFC method involves two key components: Image Bank Re-sampling (IBR) and Multi-Scaled Distribution-Weighted (MSDW) consistency loss. IBR increases the sampling frequency of tail classes, while MSDW loss adjusts the consistency between CAMs generated by classifier weights and class prototypes. The MSDW loss is weighted by the sample number gaps between classes, ensuring that tail classes receive more attention during training. The proposed method improves CAM boundaries and achieves state-of-the-art results on Pascal VOC 2012 and COCO 2014 datasets. The SFC method enhances the performance of weakly supervised semantic segmentation by addressing the over-/under-activation issues caused by long-tailed data distributions. The final CAM is obtained by combining the classifier weight CAM and prototype CAM, which jointly solves the over-/under-activation problem. The paper also provides a detailed analysis of how shared features in classifier weights lead to over-/under-activation issues under long-tailed scenarios. Theoretical analysis shows that the shared feature component in head-class classifier weights tends to be positive, while that in tail-class classifier weights tends to be negative. This leads to over-activation of head classes and under-activation of tail classes in CAMs. The SFC method is evaluated on two benchmark datasets, Pascal VOC 2012 and COCO 2014, and achieves significant improvements in segmentation performance. The results show that SFC outperforms existing methods in terms of mIoU scores, demonstrating its effectiveness in improving weakly supervised semantic segmentation. The method is also validated through ablation studies, showing that the combination of IBR and MSDW loss is crucial for achieving the best performance.
Reach us at info@study.space