First Order Motion Model for Image Animation

First Order Motion Model for Image Animation

1 Oct 2020 | Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, Nicu Sebe
The paper presents a novel approach for image animation, which involves generating a video sequence by animating an object in a source image according to the motion of a driving video. The method does not require any annotation or prior information about the specific object to be animated, making it applicable to a wide range of object categories. The framework decouples appearance and motion information using a self-supervised formulation, where motion is represented by a set of learned keypoints along with their local affine transformations. A generator network combines the appearance from the source image and the motion derived from the driving video, while also handling occlusions. The method is evaluated on various benchmarks and object categories, showing superior performance compared to state-of-the-art methods. The authors also introduce a new high-resolution dataset, Thai-Chi-HD, which can serve as a reference benchmark for future research in image animation and video generation.The paper presents a novel approach for image animation, which involves generating a video sequence by animating an object in a source image according to the motion of a driving video. The method does not require any annotation or prior information about the specific object to be animated, making it applicable to a wide range of object categories. The framework decouples appearance and motion information using a self-supervised formulation, where motion is represented by a set of learned keypoints along with their local affine transformations. A generator network combines the appearance from the source image and the motion derived from the driving video, while also handling occlusions. The method is evaluated on various benchmarks and object categories, showing superior performance compared to state-of-the-art methods. The authors also introduce a new high-resolution dataset, Thai-Chi-HD, which can serve as a reference benchmark for future research in image animation and video generation.
Reach us at info@study.space