The paper "Follow-Your-Click: Open-domain Regional Image Animation via Short Prompts" introduces a novel framework for generating regionally aware image animations using simple user interactions. The framework, named Follow-Your-Click, allows users to specify which parts of an image should move and how they should move through a single click and a short motion prompt. Key contributions include:
1. **First-Frame Masking**: This strategy significantly improves video generation quality by enhancing the model's ability to learn temporal correlations.
2. **Motion-Augmented Module**: Equipped with a short motion prompt dataset, this module enhances the model's response to short motion prompts, improving short prompt following abilities.
3. **Flow-Based Motion Magnitude Control**: This method accurately controls the speed of target movements, allowing for precise control over the intensity of motion.
The framework is evaluated on various metrics and compared with multiple baselines, including commercial tools and research methods. Extensive experiments demonstrate that Follow-Your-Click achieves superior performance in terms of user control, generation quality, and temporal coherence. The paper also discusses limitations and future directions, highlighting the potential for further improvements in handling complex and large-scale motions.The paper "Follow-Your-Click: Open-domain Regional Image Animation via Short Prompts" introduces a novel framework for generating regionally aware image animations using simple user interactions. The framework, named Follow-Your-Click, allows users to specify which parts of an image should move and how they should move through a single click and a short motion prompt. Key contributions include:
1. **First-Frame Masking**: This strategy significantly improves video generation quality by enhancing the model's ability to learn temporal correlations.
2. **Motion-Augmented Module**: Equipped with a short motion prompt dataset, this module enhances the model's response to short motion prompts, improving short prompt following abilities.
3. **Flow-Based Motion Magnitude Control**: This method accurately controls the speed of target movements, allowing for precise control over the intensity of motion.
The framework is evaluated on various metrics and compared with multiple baselines, including commercial tools and research methods. Extensive experiments demonstrate that Follow-Your-Click achieves superior performance in terms of user control, generation quality, and temporal coherence. The paper also discusses limitations and future directions, highlighting the potential for further improvements in handling complex and large-scale motions.