15 Jan 2024 | Ho Hin Lee*, Yu Gu*, Theodore Zhao, Yanbo Xu, Jianwei Yang, Naoto Usuyama, Cliff Wong, Mu Wei, Bennett A. Landman, Yuankai Huo, Alberto Santamaria-Pang, Hoifung Poon
This survey explores the application of the Segment Anything Model (SAM) in biomedical image segmentation, focusing on its adaptation and performance in medical imaging. SAM, originally developed for general-purpose computer vision, has shown significant potential in medical image processing, particularly in zero-shot learning scenarios. The study covers a six-month period from April 1, 2023, to September 30, 2023, analyzing SAM's performance across 33 open datasets. While SAM excels in many applications, it faces challenges in segmenting complex anatomical structures such as the carotid artery, adrenal glands, optic nerve, and mandible bone. The survey highlights SAM's adaptability, including its ability to adjust segmentation based on resolution scale or area of interest, and its integration with various medical imaging modalities. SAM's architecture includes an image encoder, prompt encoder, and lightweight mask decoder, enabling efficient segmentation. The study also discusses the challenges of adapting SAM to medical imaging, including domain-specific tuning, 3D imaging modalities, and knowledge distillation. SAM's performance is evaluated across different imaging modalities and tasks, with a focus on radiology, pathology, and camera imaging. The survey identifies limitations in SAM's generalization and the need for specialized fine-tuning to address medical imaging challenges. Overall, SAM represents a transformative technology in biomedical imaging, offering new opportunities for interdisciplinary collaboration and advancing the field of medical research.This survey explores the application of the Segment Anything Model (SAM) in biomedical image segmentation, focusing on its adaptation and performance in medical imaging. SAM, originally developed for general-purpose computer vision, has shown significant potential in medical image processing, particularly in zero-shot learning scenarios. The study covers a six-month period from April 1, 2023, to September 30, 2023, analyzing SAM's performance across 33 open datasets. While SAM excels in many applications, it faces challenges in segmenting complex anatomical structures such as the carotid artery, adrenal glands, optic nerve, and mandible bone. The survey highlights SAM's adaptability, including its ability to adjust segmentation based on resolution scale or area of interest, and its integration with various medical imaging modalities. SAM's architecture includes an image encoder, prompt encoder, and lightweight mask decoder, enabling efficient segmentation. The study also discusses the challenges of adapting SAM to medical imaging, including domain-specific tuning, 3D imaging modalities, and knowledge distillation. SAM's performance is evaluated across different imaging modalities and tasks, with a focus on radiology, pathology, and camera imaging. The survey identifies limitations in SAM's generalization and the need for specialized fine-tuning to address medical imaging challenges. Overall, SAM represents a transformative technology in biomedical imaging, offering new opportunities for interdisciplinary collaboration and advancing the field of medical research.