**Stable Video 4D (SV4D)** is a novel latent video diffusion model designed to generate multi-frame and multi-view consistent dynamic 3D content. Unlike previous methods that separately train generative models for video generation and novel view synthesis, SV4D uses a unified diffusion model to generate novel views of dynamic 3D objects from a single monocular reference video. The model generates temporally consistent novel views for each video frame and uses these views to optimize an implicit 4D representation (dynamic NeRF) efficiently, without the need for time-consuming SDS-based optimization. **Key Contributions:** 1. **Unified Model:** SV4D combines the advantages of both video and multi-view diffusion models to generate consistent multi-frame and multi-view videos. 2. **Efficient Optimization:** The generated novel view videos are used to optimize a 4D representation, leveraging pre-trained models and a curated 4D dataset. 3. **State-of-the-Art Performance:** Extensive experiments on synthetic and real-world datasets demonstrate SV4D's superior performance in novel view video synthesis and 4D generation. **Methodology:** - **Problem Setting:** Given a monocular input video and a user-specified camera trajectory, SV4D generates a grid of images (V × F) for each camera view. - **Network Architecture:** SV4D is built on the Stable Video Diffusion (SVD) model with additional view and frame attention blocks to ensure dynamic and multi-view consistency. - **Training:** SV4D is trained on a curated 4D dataset, ObjaverseDy, which includes dynamic 3D objects from the Objaverse dataset. - **Inference:** A mixed-sampling scheme is proposed to handle long input videos while maintaining consistency in the output image grid. **Evaluation:** - **Quantitative Results:** SV4D outperforms state-of-the-art methods in terms of video frame consistency, multi-view consistency, and 4D generation quality. - **Visual Comparison:** User studies and visual comparisons show that SV4D generates more stable, realistic, and detailed multi-view videos and 4D outputs. **Conclusion:** SV4D provides a robust foundation for dynamic 3D object generation, offering improved multi-frame and multi-view consistency and generalizability to real-world videos.**Stable Video 4D (SV4D)** is a novel latent video diffusion model designed to generate multi-frame and multi-view consistent dynamic 3D content. Unlike previous methods that separately train generative models for video generation and novel view synthesis, SV4D uses a unified diffusion model to generate novel views of dynamic 3D objects from a single monocular reference video. The model generates temporally consistent novel views for each video frame and uses these views to optimize an implicit 4D representation (dynamic NeRF) efficiently, without the need for time-consuming SDS-based optimization. **Key Contributions:** 1. **Unified Model:** SV4D combines the advantages of both video and multi-view diffusion models to generate consistent multi-frame and multi-view videos. 2. **Efficient Optimization:** The generated novel view videos are used to optimize a 4D representation, leveraging pre-trained models and a curated 4D dataset. 3. **State-of-the-Art Performance:** Extensive experiments on synthetic and real-world datasets demonstrate SV4D's superior performance in novel view video synthesis and 4D generation. **Methodology:** - **Problem Setting:** Given a monocular input video and a user-specified camera trajectory, SV4D generates a grid of images (V × F) for each camera view. - **Network Architecture:** SV4D is built on the Stable Video Diffusion (SVD) model with additional view and frame attention blocks to ensure dynamic and multi-view consistency. - **Training:** SV4D is trained on a curated 4D dataset, ObjaverseDy, which includes dynamic 3D objects from the Objaverse dataset. - **Inference:** A mixed-sampling scheme is proposed to handle long input videos while maintaining consistency in the output image grid. **Evaluation:** - **Quantitative Results:** SV4D outperforms state-of-the-art methods in terms of video frame consistency, multi-view consistency, and 4D generation quality. - **Visual Comparison:** User studies and visual comparisons show that SV4D generates more stable, realistic, and detailed multi-view videos and 4D outputs. **Conclusion:** SV4D provides a robust foundation for dynamic 3D object generation, offering improved multi-frame and multi-view consistency and generalizability to real-world videos.