ReVideo: Remake a Video with Motion and Content Control

ReVideo: Remake a Video with Motion and Content Control

22 May 2024 | Chong Mou, Mingdeng Cao, Xintao Wang, Zhaoyang Zhang, Ying Shan, Jian Zhang
ReVideo is a novel method for precise and localized video editing, allowing users to modify both visual content and motion in specific areas of a video. The method addresses the challenge of coupling and training imbalance between content and motion control by employing a three-stage training strategy that progressively decouples these aspects from coarse to fine. Additionally, a spatiotemporal adaptive fusion module integrates content and motion control across different sampling steps and spatial locations. Extensive experiments demonstrate that ReVideo performs well in various precise video editing applications, including changing content while keeping motion constant, customizing new motion trajectories, and modifying both content and motion trajectories. The method can also seamlessly extend to multi-area editing without specific training, showcasing its flexibility and robustness.ReVideo is a novel method for precise and localized video editing, allowing users to modify both visual content and motion in specific areas of a video. The method addresses the challenge of coupling and training imbalance between content and motion control by employing a three-stage training strategy that progressively decouples these aspects from coarse to fine. Additionally, a spatiotemporal adaptive fusion module integrates content and motion control across different sampling steps and spatial locations. Extensive experiments demonstrate that ReVideo performs well in various precise video editing applications, including changing content while keeping motion constant, customizing new motion trajectories, and modifying both content and motion trajectories. The method can also seamlessly extend to multi-area editing without specific training, showcasing its flexibility and robustness.
Reach us at info@study.space
Understanding ReVideo%3A Remake a Video with Motion and Content Control