First Order Motion Model for Image Animation

First Order Motion Model for Image Animation

1 Oct 2020 | Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, Nicu Sebe
This paper presents a novel first-order motion model for image animation. The method generates video sequences by animating objects in a source image based on the motion of a driving video. The framework does not require any annotations or prior knowledge about the specific object to animate. It uses a self-supervised approach to separate appearance and motion information. A generator network models occlusions and combines the appearance from the source image with motion from the driving video. The method outperforms state-of-the-art approaches on various benchmarks and object categories. It uses a set of learned keypoints and local affine transformations to model complex motions. An occlusion-aware generator is introduced to handle occlusions. The method also extends the equivariance loss to improve local affine transformation estimation. The framework is evaluated on multiple datasets, including Tai-Chi-HD, and shows significant improvements in image animation quality. The method is able to handle high-resolution datasets where other approaches generally fail. The authors also release a new high-resolution dataset, Tai-Chi-HD, which could become a reference benchmark for image animation and video generation.This paper presents a novel first-order motion model for image animation. The method generates video sequences by animating objects in a source image based on the motion of a driving video. The framework does not require any annotations or prior knowledge about the specific object to animate. It uses a self-supervised approach to separate appearance and motion information. A generator network models occlusions and combines the appearance from the source image with motion from the driving video. The method outperforms state-of-the-art approaches on various benchmarks and object categories. It uses a set of learned keypoints and local affine transformations to model complex motions. An occlusion-aware generator is introduced to handle occlusions. The method also extends the equivariance loss to improve local affine transformation estimation. The framework is evaluated on multiple datasets, including Tai-Chi-HD, and shows significant improvements in image animation quality. The method is able to handle high-resolution datasets where other approaches generally fail. The authors also release a new high-resolution dataset, Tai-Chi-HD, which could become a reference benchmark for image animation and video generation.
Reach us at info@study.space
[slides and audio] First Order Motion Model for Image Animation