VideoMV: Consistent Multi-View Generation Based on Large Video Generative Model

VideoMV: Consistent Multi-View Generation Based on Large Video Generative Model

18 Mar 2024 | Qi Zuo1*, Xiaodong Gu1*, Lingteng Qiu2,1, Yuan Dong1, Zhengyi Zhao1, Weihao Yuan1, Rui Peng4, Siyu Zhu3, Zilong Dong1, Liefeng Bo1, and Qixing Huang5
The paper introduces VideoMV, a novel framework for generating multi-view images based on text or single-image prompts. Unlike previous approaches that use 2D diffusion models, VideoMV leverages off-the-shelf video generative models fine-tuned from object-centric videos. This approach addresses two key challenges: using appropriate data for training and ensuring multi-view consistency. VideoMV introduces a dense consistent multi-view generation model and a 3D-Aware Denoising Sampling strategy to enhance multi-view consistency. The method outperforms state-of-the-art approaches in terms of efficiency and quality, generating 24 dense views with faster convergence and comparable visual quality and consistency. The project page is available at [age3d.github.io/VideoMV](https://age3d.github.io/VideoMV).The paper introduces VideoMV, a novel framework for generating multi-view images based on text or single-image prompts. Unlike previous approaches that use 2D diffusion models, VideoMV leverages off-the-shelf video generative models fine-tuned from object-centric videos. This approach addresses two key challenges: using appropriate data for training and ensuring multi-view consistency. VideoMV introduces a dense consistent multi-view generation model and a 3D-Aware Denoising Sampling strategy to enhance multi-view consistency. The method outperforms state-of-the-art approaches in terms of efficiency and quality, generating 24 dense views with faster convergence and comparable visual quality and consistency. The project page is available at [age3d.github.io/VideoMV](https://age3d.github.io/VideoMV).
Reach us at info@study.space
Understanding VideoMV%3A Consistent Multi-View Generation Based on Large Video Generative Model