26 Feb 2024 | Zhen Chen, Qing Xu, Xinyu Liu, and Yixuan Yuan, Member, IEEE
The paper introduces the Universal Prompt-Free SAM (UN-SAM) framework for nuclei image segmentation in digital pathology. UN-SAM addresses the challenges of precise nuclei segmentation, which is crucial for morphological quantification and tumor grade assessment. Traditional methods often rely on labor-intensive manual annotations, making them impractical for large-scale clinical applications. To overcome this, UN-SAM employs a multi-scale Self-Prompt Generation (SPGen) module to automatically generate high-quality mask hints, eliminating the need for manual prompts. Additionally, the Domain-adaptive Tuning Encoder (DT-Encoder) and Domain Query-enhanced Decoder (DQ-Decoder) are designed to enhance the generalization capabilities of the SAM model across different nuclei domains. Extensive experiments on various datasets demonstrate that UN-SAM outperforms state-of-the-art methods in both instance and semantic segmentation tasks, especially in zero-shot scenarios. The proposed framework streamlines clinical workflows and enhances the efficiency of nuclei image segmentation, making it a promising solution for digital pathology.The paper introduces the Universal Prompt-Free SAM (UN-SAM) framework for nuclei image segmentation in digital pathology. UN-SAM addresses the challenges of precise nuclei segmentation, which is crucial for morphological quantification and tumor grade assessment. Traditional methods often rely on labor-intensive manual annotations, making them impractical for large-scale clinical applications. To overcome this, UN-SAM employs a multi-scale Self-Prompt Generation (SPGen) module to automatically generate high-quality mask hints, eliminating the need for manual prompts. Additionally, the Domain-adaptive Tuning Encoder (DT-Encoder) and Domain Query-enhanced Decoder (DQ-Decoder) are designed to enhance the generalization capabilities of the SAM model across different nuclei domains. Extensive experiments on various datasets demonstrate that UN-SAM outperforms state-of-the-art methods in both instance and semantic segmentation tasks, especially in zero-shot scenarios. The proposed framework streamlines clinical workflows and enhances the efficiency of nuclei image segmentation, making it a promising solution for digital pathology.