ReVideo: Remake a Video with Motion and Content Control

ReVideo: Remake a Video with Motion and Content Control

22 May 2024 | Chong Mou, Mingdeng Cao, Xintao Wang, Zhaoyang Zhang, Ying Shan, Jian Zhang
ReVideo is a novel method for local video editing that allows precise modification of both content and motion in specific areas. The method enables users to edit video content by modifying the first frame and using trajectory lines as motion control signals. It addresses the challenge of coupling and training imbalance between content and motion control by developing a three-stage training strategy that progressively decouples these aspects from coarse to fine. Additionally, a spatiotemporal adaptive fusion module is introduced to integrate content and motion control across various sampling steps and spatial locations. Extensive experiments show that ReVideo performs well in several video editing applications, including changing content while keeping motion constant, customizing new motion trajectories, and modifying both content and motion. The method can also be extended to multi-area editing without specific training, demonstrating its flexibility and robustness. ReVideo is not limited to single-region editing and can customize multiple areas in parallel. The method is evaluated against existing methods, showing superior performance in terms of video quality and editing accuracy. The method is also compared with other related works, demonstrating its effectiveness in local video editing. The method is trained on the WebVid dataset and uses a stable video diffusion model as the base model. The method is evaluated using automatic metrics and human evaluation, showing that ReVideo performs better than other methods in most evaluation terms. The method is also compared with other methods in terms of text alignment and human evaluation, showing that ReVideo has significant advantages over other methods in text-guided local editing. The method is also compared with other methods in terms of motion control, showing that ReVideo can effectively broadcast edited content throughout the entire video while allowing users to customize the motion in editing areas. The method is also compared with other methods in terms of motion control, showing that ReVideo can effectively broadcast edited content throughout the entire video while allowing users to customize the motion in editing areas.ReVideo is a novel method for local video editing that allows precise modification of both content and motion in specific areas. The method enables users to edit video content by modifying the first frame and using trajectory lines as motion control signals. It addresses the challenge of coupling and training imbalance between content and motion control by developing a three-stage training strategy that progressively decouples these aspects from coarse to fine. Additionally, a spatiotemporal adaptive fusion module is introduced to integrate content and motion control across various sampling steps and spatial locations. Extensive experiments show that ReVideo performs well in several video editing applications, including changing content while keeping motion constant, customizing new motion trajectories, and modifying both content and motion. The method can also be extended to multi-area editing without specific training, demonstrating its flexibility and robustness. ReVideo is not limited to single-region editing and can customize multiple areas in parallel. The method is evaluated against existing methods, showing superior performance in terms of video quality and editing accuracy. The method is also compared with other related works, demonstrating its effectiveness in local video editing. The method is trained on the WebVid dataset and uses a stable video diffusion model as the base model. The method is evaluated using automatic metrics and human evaluation, showing that ReVideo performs better than other methods in most evaluation terms. The method is also compared with other methods in terms of text alignment and human evaluation, showing that ReVideo has significant advantages over other methods in text-guided local editing. The method is also compared with other methods in terms of motion control, showing that ReVideo can effectively broadcast edited content throughout the entire video while allowing users to customize the motion in editing areas. The method is also compared with other methods in terms of motion control, showing that ReVideo can effectively broadcast edited content throughout the entire video while allowing users to customize the motion in editing areas.
Reach us at info@study.space
[slides] ReVideo%3A Remake a Video with Motion and Content Control | StudySpace