26 Feb 2024 | Zhen Chen, Qing Xu, Xinyu Liu, and Yixuan Yuan, Member, IEEE
UN-SAM is a universal prompt-free segmentation framework for nuclei images, designed to overcome the limitations of existing segmentation methods in digital pathology. The framework eliminates the need for manual annotations by using a multi-scale Self-Prompt Generation (SPGen) module to automatically generate high-quality mask hints for nuclei segmentation. Additionally, a Domain-adaptive Tuning Encoder (DT-Encoder) and Domain Query-enhanced Decoder (DQ-Decoder) are introduced to enhance the generalization capability of the model across various nuclei image domains. The DT-Encoder harmonizes visual features with domain-specific knowledge, while the DQ-Decoder leverages learnable domain queries to improve segmentation accuracy. Extensive experiments show that UN-SAM outperforms state-of-the-art methods in nuclei instance and semantic segmentation, particularly in zero-shot scenarios. The framework achieves superior performance across diverse nuclei image datasets, demonstrating its effectiveness in clinical applications. The source code is available at https://github.com/CUHK-AIM-Group/UN-SAM.UN-SAM is a universal prompt-free segmentation framework for nuclei images, designed to overcome the limitations of existing segmentation methods in digital pathology. The framework eliminates the need for manual annotations by using a multi-scale Self-Prompt Generation (SPGen) module to automatically generate high-quality mask hints for nuclei segmentation. Additionally, a Domain-adaptive Tuning Encoder (DT-Encoder) and Domain Query-enhanced Decoder (DQ-Decoder) are introduced to enhance the generalization capability of the model across various nuclei image domains. The DT-Encoder harmonizes visual features with domain-specific knowledge, while the DQ-Decoder leverages learnable domain queries to improve segmentation accuracy. Extensive experiments show that UN-SAM outperforms state-of-the-art methods in nuclei instance and semantic segmentation, particularly in zero-shot scenarios. The framework achieves superior performance across diverse nuclei image datasets, demonstrating its effectiveness in clinical applications. The source code is available at https://github.com/CUHK-AIM-Group/UN-SAM.