07/2024 | Weiming Ren, Huan Yang, Ge Zhang, Cong Wei, Xinrun Du, Wenhao Huang, Wenhui Chen
The paper "ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation" addresses the challenge of maintaining visual consistency in image-to-video (I2V) generation. The authors propose CONSISTI2V, a diffusion-based method that enhances visual consistency by introducing spatiotemporal attention over the first frame and noise initialization from the low-frequency band of the first frame. These techniques ensure spatial and motion consistency, as well as layout consistency, resulting in highly consistent videos. The method is evaluated using a comprehensive benchmark, I2V-Bench, and shown to outperform existing methods in terms of visual quality and consistency. The paper also discusses the potential applications of CONSISTI2V in autoregressive long video generation and camera motion control.The paper "ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation" addresses the challenge of maintaining visual consistency in image-to-video (I2V) generation. The authors propose CONSISTI2V, a diffusion-based method that enhances visual consistency by introducing spatiotemporal attention over the first frame and noise initialization from the low-frequency band of the first frame. These techniques ensure spatial and motion consistency, as well as layout consistency, resulting in highly consistent videos. The method is evaluated using a comprehensive benchmark, I2V-Bench, and shown to outperform existing methods in terms of visual quality and consistency. The paper also discusses the potential applications of CONSISTI2V in autoregressive long video generation and camera motion control.