1 Nov 2024 | Desai Xie, Sai Bi, Zhixin Shu, Kai Zhang, Zexiang Xu, Yi Zhou, Sören Pirk, Arie Kaufman, Xin Sun, Hao Tan
The paper introduces *LRM-Zero*, a Large Reconstruction Model (LRM) trained entirely on synthesized 3D data, specifically *Zeroverse*, to achieve high-quality sparse-view 3D reconstruction. *Zeroverse* is a procedural dataset created by automatically synthesizing 3D objects from simple primitive shapes with random texturing and augmentations, such as height fields, boolean differences, and wireframes. Unlike datasets like Objaverse, which are captured or crafted by humans to approximate real 3D data, *Zeroverse* completely ignores realistic global semantics but is rich in complex geometric and texture details. The paper demonstrates that *LRM-Zero*, trained with *Zeroverse*, can achieve visually competitive results with models trained on Objaverse, showing that 3D reconstruction can rely more on local information than global semantics. The study also analyzes the effectiveness of *Zeroverse*'s design choices, particularly shape augmentations, and their impact on training stability. The work highlights the potential of using synthetic data to address the scarcity, licensing, and bias issues in 3D data, and provides a proof-of-concept for using synthesized data in 3D reconstruction research.The paper introduces *LRM-Zero*, a Large Reconstruction Model (LRM) trained entirely on synthesized 3D data, specifically *Zeroverse*, to achieve high-quality sparse-view 3D reconstruction. *Zeroverse* is a procedural dataset created by automatically synthesizing 3D objects from simple primitive shapes with random texturing and augmentations, such as height fields, boolean differences, and wireframes. Unlike datasets like Objaverse, which are captured or crafted by humans to approximate real 3D data, *Zeroverse* completely ignores realistic global semantics but is rich in complex geometric and texture details. The paper demonstrates that *LRM-Zero*, trained with *Zeroverse*, can achieve visually competitive results with models trained on Objaverse, showing that 3D reconstruction can rely more on local information than global semantics. The study also analyzes the effectiveness of *Zeroverse*'s design choices, particularly shape augmentations, and their impact on training stability. The work highlights the potential of using synthetic data to address the scarcity, licensing, and bias issues in 3D data, and provides a proof-of-concept for using synthesized data in 3D reconstruction research.