Endora is an innovative approach to generate medical videos that simulate clinical endoscopy scenes. The paper introduces a novel generative model design that integrates a meticulously crafted spatial-temporal video transformer with advanced 2D vision foundation model priors, explicitly modeling spatial-temporal dynamics during video generation. The authors also pioneer the first public benchmark for endoscopy simulation with video generation models, adapting existing state-of-the-art methods for this endeavor. Endora demonstrates exceptional visual quality in generating endoscopy videos, surpassing state-of-the-art methods in extensive testing. Moreover, the paper explores how this endoscopy simulator can empower downstream video analysis tasks and even generate 3D medical scenes with multi-view consistency. Endora marks a notable breakthrough in the deployment of generative AI for clinical endoscopy research, setting a substantial stage for further advances in medical content generation. The paper presents a framework to generate spatially and temporally coherent and plausible endoscopy videos to synthesize realistic clinical scenes. The authors introduce a diffusion model for video generation, which is trained to generate plausible endoscopy videos given the collection of actual clinical observations. The model integrates an advanced video transformer architecture with a latent diffusion model, facilitating the extraction of long-range correlations in terms of both spatial and temporal dimension from video data. The model also incorporates a prior from a 2D foundation model, DINO, to guide feature extraction. The authors conduct comprehensive experiments on three public endoscopy video datasets, demonstrating that Endora excels over state-of-the-art methods in terms of visual fidelity and performance in downstream video analysis tasks. The results show that Endora can produce highly realistic endoscopic videos, showcasing its effectiveness and potential for medical video generation with rich dynamics. The paper also explores the potential of Endora as a surgical world simulator, demonstrating its ability to generate 3D surgical scenes with multi-view consistency. The authors conclude that Endora represents a significant advancement in the application of generative AI for clinical endoscopy research, providing key insights and setting a strong foundation for future research on medical generated content.Endora is an innovative approach to generate medical videos that simulate clinical endoscopy scenes. The paper introduces a novel generative model design that integrates a meticulously crafted spatial-temporal video transformer with advanced 2D vision foundation model priors, explicitly modeling spatial-temporal dynamics during video generation. The authors also pioneer the first public benchmark for endoscopy simulation with video generation models, adapting existing state-of-the-art methods for this endeavor. Endora demonstrates exceptional visual quality in generating endoscopy videos, surpassing state-of-the-art methods in extensive testing. Moreover, the paper explores how this endoscopy simulator can empower downstream video analysis tasks and even generate 3D medical scenes with multi-view consistency. Endora marks a notable breakthrough in the deployment of generative AI for clinical endoscopy research, setting a substantial stage for further advances in medical content generation. The paper presents a framework to generate spatially and temporally coherent and plausible endoscopy videos to synthesize realistic clinical scenes. The authors introduce a diffusion model for video generation, which is trained to generate plausible endoscopy videos given the collection of actual clinical observations. The model integrates an advanced video transformer architecture with a latent diffusion model, facilitating the extraction of long-range correlations in terms of both spatial and temporal dimension from video data. The model also incorporates a prior from a 2D foundation model, DINO, to guide feature extraction. The authors conduct comprehensive experiments on three public endoscopy video datasets, demonstrating that Endora excels over state-of-the-art methods in terms of visual fidelity and performance in downstream video analysis tasks. The results show that Endora can produce highly realistic endoscopic videos, showcasing its effectiveness and potential for medical video generation with rich dynamics. The paper also explores the potential of Endora as a surgical world simulator, demonstrating its ability to generate 3D surgical scenes with multi-view consistency. The authors conclude that Endora represents a significant advancement in the application of generative AI for clinical endoscopy research, providing key insights and setting a strong foundation for future research on medical generated content.