14 Aug 2024 | Zijie Wu, Chaohui Yu, Yanqin Jiang, Chenjie Cao, Fan Wang, Xiang Bai
The paper introduces SC4D, a novel framework for generating dynamic 3D objects from a single-view video. SC4D decouples the motion and appearance of the objects using sparse control points and dense 3D Gaussians, addressing the challenges of reference view alignment, spatio-temporal consistency, and motion fidelity. The method includes Adaptive Gaussian (AG) initialization and Gaussian Alignment (GA) loss to mitigate shape degeneration issues, ensuring accurate motion and shape learning. Comprehensive evaluations show that SC4D outperforms existing methods in quality and efficiency. Additionally, SC4D enables a novel application for motion transfer, where learned motions are applied to diverse 4D entities based on textual descriptions. The paper also discusses related work, experimental settings, and limitations, concluding with a discussion of future directions.The paper introduces SC4D, a novel framework for generating dynamic 3D objects from a single-view video. SC4D decouples the motion and appearance of the objects using sparse control points and dense 3D Gaussians, addressing the challenges of reference view alignment, spatio-temporal consistency, and motion fidelity. The method includes Adaptive Gaussian (AG) initialization and Gaussian Alignment (GA) loss to mitigate shape degeneration issues, ensuring accurate motion and shape learning. Comprehensive evaluations show that SC4D outperforms existing methods in quality and efficiency. Additionally, SC4D enables a novel application for motion transfer, where learned motions are applied to diverse 4D entities based on textual descriptions. The paper also discusses related work, experimental settings, and limitations, concluding with a discussion of future directions.