This paper presents REACTO, a method for reconstructing general articulated 3D objects from a single casual video. The key challenge is to accurately model the shape, texture, and motion of objects with multiple rigid components that are often adjacent in their usual poses. Existing methods, such as BANMo, MoDA, and PPR, face difficulties in modeling such objects due to issues with deformation and joint handling. To address these challenges, the authors propose Quasi-Rigid Blend Skinning (QRBS), a novel deformation model that combines the rigidity of Rigid Skinning with the flexibility of Dual Quaternion Blend Skinning. QRBS uses quasi-sparse skinning weights and geodesic point assignment to achieve precise motion reconstruction.
The method employs a canonical Neural Radiance Field (NeRF) for shape and appearance modeling, and a deformation model to transform 3D points between observation and canonical spaces. The deformation model is optimized using volume rendering and various reconstruction losses, including color, object mask, optical flow, and pixel features. The method is evaluated on both real-world and synthetic datasets, demonstrating superior performance in terms of fidelity and detail compared to existing methods.
The authors also conduct an ablation study comparing their method with other deformation models, such as displacement fields and invertible Real-NVP. The results show that QRBS consistently outperforms these methods in both synthetic and real-world scenarios. Additionally, the method is compared with state-of-the-art methods, including BANMo, MoDA, and PPR, and is found to produce more accurate and detailed reconstructions of articulated objects.
The method is particularly effective in handling objects with multiple rigid components that are adjacent in their usual poses, such as scissors, staplers, and faucets. The use of geodesic point assignment helps prevent surface tearing and ensures smooth deformation. The quasi-sparse skinning weights also help maintain the rigidity of each component while allowing for flexible deformation at the joints.
Overall, REACTO demonstrates superior performance in reconstructing general articulated 3D objects from a single casual video, achieving enhanced modeling and precision by redefining rigging structures and employing Quasi-Rigid Blend Skinning. The method is effective in both real-world and synthetic datasets, and the results show that it outperforms existing methods in terms of fidelity and detail.This paper presents REACTO, a method for reconstructing general articulated 3D objects from a single casual video. The key challenge is to accurately model the shape, texture, and motion of objects with multiple rigid components that are often adjacent in their usual poses. Existing methods, such as BANMo, MoDA, and PPR, face difficulties in modeling such objects due to issues with deformation and joint handling. To address these challenges, the authors propose Quasi-Rigid Blend Skinning (QRBS), a novel deformation model that combines the rigidity of Rigid Skinning with the flexibility of Dual Quaternion Blend Skinning. QRBS uses quasi-sparse skinning weights and geodesic point assignment to achieve precise motion reconstruction.
The method employs a canonical Neural Radiance Field (NeRF) for shape and appearance modeling, and a deformation model to transform 3D points between observation and canonical spaces. The deformation model is optimized using volume rendering and various reconstruction losses, including color, object mask, optical flow, and pixel features. The method is evaluated on both real-world and synthetic datasets, demonstrating superior performance in terms of fidelity and detail compared to existing methods.
The authors also conduct an ablation study comparing their method with other deformation models, such as displacement fields and invertible Real-NVP. The results show that QRBS consistently outperforms these methods in both synthetic and real-world scenarios. Additionally, the method is compared with state-of-the-art methods, including BANMo, MoDA, and PPR, and is found to produce more accurate and detailed reconstructions of articulated objects.
The method is particularly effective in handling objects with multiple rigid components that are adjacent in their usual poses, such as scissors, staplers, and faucets. The use of geodesic point assignment helps prevent surface tearing and ensures smooth deformation. The quasi-sparse skinning weights also help maintain the rigidity of each component while allowing for flexible deformation at the joints.
Overall, REACTO demonstrates superior performance in reconstructing general articulated 3D objects from a single casual video, achieving enhanced modeling and precision by redefining rigging structures and employing Quasi-Rigid Blend Skinning. The method is effective in both real-world and synthetic datasets, and the results show that it outperforms existing methods in terms of fidelity and detail.