ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulation
This paper proposes a dynamic Gaussian Splatting method named ManiGaussian for multi-task robotic manipulation. The method mines scene dynamics via future scene reconstruction. The dynamic Gaussian Splatting framework infers the semantics propagation in the Gaussian embedding space, where the semantic representation is leveraged to predict the optimal robot action. A Gaussian world model is built to parameterize the distribution in the dynamic Gaussian Splatting framework, providing informative supervision in the interactive environment via future scene reconstruction. The method is evaluated on 10 RLBench tasks with 166 variations, achieving a 13.1% improvement in average success rate over state-of-the-art methods.
The method addresses the challenges of language-conditioned robotic manipulation in unstructured environments by learning scene-level spatiotemporal dynamics. It leverages a dynamic Gaussian Splatting framework that models the propagation of diverse semantic features in the Gaussian embedding space. A Gaussian world model is used to parameterize distributions and provide informative supervision through future scene reconstruction. The method outperforms previous approaches by incorporating scene dynamics into the action prediction process.
The method is evaluated on 10 RLBench tasks with 166 variations, achieving a 13.1% improvement in average success rate over state-of-the-art methods. The results demonstrate the effectiveness of the dynamic Gaussian Splatting framework in learning scene dynamics for robotic manipulation. The method is able to accurately predict robot actions in unstructured environments by incorporating scene dynamics into the action prediction process. The method also shows improved performance in tasks requiring geometric reasoning and semantic understanding. The results indicate that the proposed method is effective in learning scene-level spatiotemporal dynamics for robotic manipulation tasks.ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulation
This paper proposes a dynamic Gaussian Splatting method named ManiGaussian for multi-task robotic manipulation. The method mines scene dynamics via future scene reconstruction. The dynamic Gaussian Splatting framework infers the semantics propagation in the Gaussian embedding space, where the semantic representation is leveraged to predict the optimal robot action. A Gaussian world model is built to parameterize the distribution in the dynamic Gaussian Splatting framework, providing informative supervision in the interactive environment via future scene reconstruction. The method is evaluated on 10 RLBench tasks with 166 variations, achieving a 13.1% improvement in average success rate over state-of-the-art methods.
The method addresses the challenges of language-conditioned robotic manipulation in unstructured environments by learning scene-level spatiotemporal dynamics. It leverages a dynamic Gaussian Splatting framework that models the propagation of diverse semantic features in the Gaussian embedding space. A Gaussian world model is used to parameterize distributions and provide informative supervision through future scene reconstruction. The method outperforms previous approaches by incorporating scene dynamics into the action prediction process.
The method is evaluated on 10 RLBench tasks with 166 variations, achieving a 13.1% improvement in average success rate over state-of-the-art methods. The results demonstrate the effectiveness of the dynamic Gaussian Splatting framework in learning scene dynamics for robotic manipulation. The method is able to accurately predict robot actions in unstructured environments by incorporating scene dynamics into the action prediction process. The method also shows improved performance in tasks requiring geometric reasoning and semantic understanding. The results indicate that the proposed method is effective in learning scene-level spatiotemporal dynamics for robotic manipulation tasks.