MVD²: Efficient Multiview 3D Reconstruction for Multiview Diffusion

MVD²: Efficient Multiview 3D Reconstruction for Multiview Diffusion

22 Feb 2024 | XIN-YANG ZHENG*, Tsinghua University, P. R. China; HAO PAN, Microsoft Research Asia, P. R. China; YU-XIAO GUO, Microsoft Research Asia, P. R. China; XIN TONG, Microsoft Research Asia, P. R. China; YANG LIU†, Microsoft Research Asia, P. R. China
The paper introduces MVD², an efficient 3D reconstruction method for multiview diffusion (MVD) images. MVD methods, which generate multiple views of a 3D object from text or image prompts, often struggle with the inconsistency and sparsity of the generated images, leading to low-quality 3D reconstructions. MVD² addresses these challenges by aggregating image features into a 3D feature volume and decoding them into a 3D mesh. The method is trained using a view-dependent training scheme, which ensures that the reconstructed shape aligns well with the reference view and maintains structural similarity at other views. MVD² is evaluated on various datasets and compared with other 3D generation methods, demonstrating superior performance in terms of quality and efficiency. The method is also shown to be robust to different MVD models and generalizes well to unseen data.The paper introduces MVD², an efficient 3D reconstruction method for multiview diffusion (MVD) images. MVD methods, which generate multiple views of a 3D object from text or image prompts, often struggle with the inconsistency and sparsity of the generated images, leading to low-quality 3D reconstructions. MVD² addresses these challenges by aggregating image features into a 3D feature volume and decoding them into a 3D mesh. The method is trained using a view-dependent training scheme, which ensures that the reconstructed shape aligns well with the reference view and maintains structural similarity at other views. MVD² is evaluated on various datasets and compared with other 3D generation methods, demonstrating superior performance in terms of quality and efficiency. The method is also shown to be robust to different MVD models and generalizes well to unseen data.
Reach us at info@study.space
[slides and audio] MVD2%3A Efficient Multiview 3D Reconstruction for Multiview Diffusion