GENE0H DIFFUSION: TOWARDS GENERALIZABLE HAND-OBJECT INTERACTION DENOISING VIA DE-NOISING DIFFUSION

GENE0H DIFFUSION: TOWARDS GENERALIZABLE HAND-OBJECT INTERACTION DENOISING VIA DE-NOISING DIFFUSION

2024 | Xueyi Liu, Li Yi
GeneOH Diffusion is a novel method for denoising hand-object interactions (HOI) with strong generalization capabilities. The method addresses the challenge of removing interaction artifacts from erroneous HOI sequences by incorporating two key designs: an innovative contact-centric HOI representation named GeneOH and a new domain-generalizable denoising scheme. GeneOH informatively parameterizes the HOI process, facilitating enhanced generalization across various HOI scenarios. The new denoising scheme consists of a canonical denoising model trained to project noisy data samples from a whitened noise space to a clean data manifold and a "denoising via diffusion" strategy which can handle input trajectories with various noise patterns by first diffusing them to align with the whitened noise space and cleaning via the canonical denoiser. Extensive experiments on four benchmarks with significant domain variations demonstrate the superior effectiveness of our method. GeneOH Diffusion also shows promise for various downstream applications. The method is evaluated on three datasets, GRAB, HOI4D, and ARCTIC, showing remarkable effectiveness and generalizability. The method outperforms prior arts by a significant margin, demonstrating its ability to handle novel and difficult noise patterns and challenging interactions. The method is also able to produce multiple reasonable solutions for a single input, showing its stochastic denoising capability. The method is designed to handle complex interaction noise through a progressive denoising strategy, cleaning the interaction process in three stages. The method is also able to fit a parameterized hand sequence to the denoised trajectory. The method is evaluated on four test sets with different levels of domain shift, demonstrating its denoising ability and generalization ability. The method is also able to handle challenging bimanual and dynamic interactions with changing contacts. The method is able to produce visually appealing and motion-aware results with accurate contacts. The method is also able to handle difficult shapes such as the mug handle and scissor rings which are very easy to penetrate. The method is also able to handle difficult and dynamic motions with changing contacts. The method is able to produce diverse results with discrete modes. The method is also able to refine hand trajectory estimations derived from image sequence observations. The method is also able to refine noisy retargeted hand motions. The method is able to handle complex interaction noise through a progressive denoising strategy, cleaning the interaction process in three stages. The method is also able to handle complex interaction noise through a progressive denoising strategy, cleaning the interaction process in three stages. The method is also able to handle complex interaction noise through a progressive denoising strategy, cleaning the interaction process in three stages. The method is also able to handle complex interaction noise through a progressive denoising strategy, cleaning the interaction process in three stages. The method is also able to handle complex interaction noise through a progressive denoising strategy, cleaning the interaction process in three stages. The method is also able to handle complex interaction noise through a progressiveGeneOH Diffusion is a novel method for denoising hand-object interactions (HOI) with strong generalization capabilities. The method addresses the challenge of removing interaction artifacts from erroneous HOI sequences by incorporating two key designs: an innovative contact-centric HOI representation named GeneOH and a new domain-generalizable denoising scheme. GeneOH informatively parameterizes the HOI process, facilitating enhanced generalization across various HOI scenarios. The new denoising scheme consists of a canonical denoising model trained to project noisy data samples from a whitened noise space to a clean data manifold and a "denoising via diffusion" strategy which can handle input trajectories with various noise patterns by first diffusing them to align with the whitened noise space and cleaning via the canonical denoiser. Extensive experiments on four benchmarks with significant domain variations demonstrate the superior effectiveness of our method. GeneOH Diffusion also shows promise for various downstream applications. The method is evaluated on three datasets, GRAB, HOI4D, and ARCTIC, showing remarkable effectiveness and generalizability. The method outperforms prior arts by a significant margin, demonstrating its ability to handle novel and difficult noise patterns and challenging interactions. The method is also able to produce multiple reasonable solutions for a single input, showing its stochastic denoising capability. The method is designed to handle complex interaction noise through a progressive denoising strategy, cleaning the interaction process in three stages. The method is also able to fit a parameterized hand sequence to the denoised trajectory. The method is evaluated on four test sets with different levels of domain shift, demonstrating its denoising ability and generalization ability. The method is also able to handle challenging bimanual and dynamic interactions with changing contacts. The method is able to produce visually appealing and motion-aware results with accurate contacts. The method is also able to handle difficult shapes such as the mug handle and scissor rings which are very easy to penetrate. The method is also able to handle difficult and dynamic motions with changing contacts. The method is able to produce diverse results with discrete modes. The method is also able to refine hand trajectory estimations derived from image sequence observations. The method is also able to refine noisy retargeted hand motions. The method is able to handle complex interaction noise through a progressive denoising strategy, cleaning the interaction process in three stages. The method is also able to handle complex interaction noise through a progressive denoising strategy, cleaning the interaction process in three stages. The method is also able to handle complex interaction noise through a progressive denoising strategy, cleaning the interaction process in three stages. The method is also able to handle complex interaction noise through a progressive denoising strategy, cleaning the interaction process in three stages. The method is also able to handle complex interaction noise through a progressive denoising strategy, cleaning the interaction process in three stages. The method is also able to handle complex interaction noise through a progressive
Reach us at info@study.space
[slides and audio] GeneOH Diffusion%3A Towards Generalizable Hand-Object Interaction Denoising via Denoising Diffusion