Grounded SAM is a novel open-vocabulary detection and segmentation model that integrates Grounding DINO with the Segment Anything Model (SAM). It enables the detection and segmentation of any regions in images based on arbitrary text inputs, facilitating a wide range of visual tasks. Grounded SAM can be seamlessly integrated with other open-world models to perform complex visual tasks. For example, when combined with Stable-Diffusion, it enables highly controllable image editing, and when integrated with OSX, it allows for promptable 3D human motion analysis. Grounded SAM also achieves superior performance on open-vocabulary benchmarks, achieving a mean AP of 48.7 on the SegInW zero-shot benchmark with the combination of Grounding DINO-Base and SAM-Huge models. The paper introduces Grounded SAM as an innovative approach within the Ensemble Foundation Models framework, combining open-set detector models like Grounding DINO with promptable segmentation models like SAM. This approach effectively tackles open-set segmentation by dividing it into two components: open-set detection and promptable segmentation. Grounded SAM offers a powerful and comprehensive platform that further facilitates the efficient fusion of different expert models to tackle more intricate open-world tasks. Grounded SAM can be combined with other open-world models to achieve various applications. For instance, when combined with Recognize Anything (RAM), the RAM-Grounded-SAM model can automatically identify and segment things or objects within images without the need for any textual input, facilitating automatic image annotation tasks. Similarly, Grounded SAM, when coupled with the inpainting capability of Stable Diffusion, can execute highly precise image editing tasks. The paper also discusses various extensions of Grounded SAM, including RAM-Grounded-SAM for automatic dense image annotation, Grounded-SAM-SD for highly accurate and controllable image editing, and Grounded-SAM-OSX for promptable human motion analysis. These extensions demonstrate the versatility and effectiveness of Grounded SAM in various visual tasks. The effectiveness of Grounded SAM is validated on the Segmentation in the Wild (SGinW) zero-shot benchmark, where it achieves significant performance improvements compared to previously unified open-set segmentation models. The results show that the combination of Grounding DINO Base and Large Model with SAM-Huge results in significant performance improvements in the zero-shot settings of SGinW. The paper concludes that Grounded SAM and its extensions offer a powerful and flexible platform for various vision tasks, enabling the seamless integration of diverse expert models to accomplish complex open-world tasks. The methodology has significant prospects, including establishing a closed loop between annotation data and model training, and combining with Large Language Models (LLMs) for effective execution of computer vision tasks. The contributions of the paper are acknowledged, and the authors express their gratitude to the research community for their support.Grounded SAM is a novel open-vocabulary detection and segmentation model that integrates Grounding DINO with the Segment Anything Model (SAM). It enables the detection and segmentation of any regions in images based on arbitrary text inputs, facilitating a wide range of visual tasks. Grounded SAM can be seamlessly integrated with other open-world models to perform complex visual tasks. For example, when combined with Stable-Diffusion, it enables highly controllable image editing, and when integrated with OSX, it allows for promptable 3D human motion analysis. Grounded SAM also achieves superior performance on open-vocabulary benchmarks, achieving a mean AP of 48.7 on the SegInW zero-shot benchmark with the combination of Grounding DINO-Base and SAM-Huge models. The paper introduces Grounded SAM as an innovative approach within the Ensemble Foundation Models framework, combining open-set detector models like Grounding DINO with promptable segmentation models like SAM. This approach effectively tackles open-set segmentation by dividing it into two components: open-set detection and promptable segmentation. Grounded SAM offers a powerful and comprehensive platform that further facilitates the efficient fusion of different expert models to tackle more intricate open-world tasks. Grounded SAM can be combined with other open-world models to achieve various applications. For instance, when combined with Recognize Anything (RAM), the RAM-Grounded-SAM model can automatically identify and segment things or objects within images without the need for any textual input, facilitating automatic image annotation tasks. Similarly, Grounded SAM, when coupled with the inpainting capability of Stable Diffusion, can execute highly precise image editing tasks. The paper also discusses various extensions of Grounded SAM, including RAM-Grounded-SAM for automatic dense image annotation, Grounded-SAM-SD for highly accurate and controllable image editing, and Grounded-SAM-OSX for promptable human motion analysis. These extensions demonstrate the versatility and effectiveness of Grounded SAM in various visual tasks. The effectiveness of Grounded SAM is validated on the Segmentation in the Wild (SGinW) zero-shot benchmark, where it achieves significant performance improvements compared to previously unified open-set segmentation models. The results show that the combination of Grounding DINO Base and Large Model with SAM-Huge results in significant performance improvements in the zero-shot settings of SGinW. The paper concludes that Grounded SAM and its extensions offer a powerful and flexible platform for various vision tasks, enabling the seamless integration of diverse expert models to accomplish complex open-world tasks. The methodology has significant prospects, including establishing a closed loop between annotation data and model training, and combining with Large Language Models (LLMs) for effective execution of computer vision tasks. The contributions of the paper are acknowledged, and the authors express their gratitude to the research community for their support.