24 Mar 2024 | Fei Wang1, Dan Guo1,2*, Kun Li1, Zhun Zhong1,3, Meng Wang1,2*
The paper introduces FD4MM, a novel framework for Video Motion Magnification (VMM) that employs Frequency Decoupling and a Multi-level Isomorphic Architecture to capture high-frequency details and stable low-frequency structures in video frames. The method aims to enhance the accuracy and quality of motion magnification by separating and amplifying subtle motion information while reducing noise and artifacts. Key contributions include:
1. **Frequency Decoupling**: The framework uses frequency decoupling to separate high- and low-frequency features, leveraging the stable energy distribution of low-frequency components for modeling the motion field.
2. **Multi-level Isomorphic Architecture**: This architecture progressively separates high-frequency components within low-frequency features, ensuring a stable structure for motion magnification.
3. **Sparse High/Low-pass Filters**: These filters are designed to mitigate the degradation of high-frequency details and low-frequency structures caused by noise, enhancing the integrity of the motion field.
4. **Sparse Frequency Mixer**: This component promotes seamless recoupling of high-frequency details and magnified low-frequency features, reducing ringing artifacts.
5. **Contrastive Regularization**: A novel loss function is introduced to strengthen the model's ability to discriminate irrelevant features, reducing undesired motion magnification.
Experiments on both real-world and synthetic datasets demonstrate that FD4MM outperforms state-of-the-art methods in terms of magnification quality, with fewer FLOPs and faster inference speed. The paper also includes detailed ablation studies and qualitative comparisons to validate the effectiveness of each component and the overall framework.The paper introduces FD4MM, a novel framework for Video Motion Magnification (VMM) that employs Frequency Decoupling and a Multi-level Isomorphic Architecture to capture high-frequency details and stable low-frequency structures in video frames. The method aims to enhance the accuracy and quality of motion magnification by separating and amplifying subtle motion information while reducing noise and artifacts. Key contributions include:
1. **Frequency Decoupling**: The framework uses frequency decoupling to separate high- and low-frequency features, leveraging the stable energy distribution of low-frequency components for modeling the motion field.
2. **Multi-level Isomorphic Architecture**: This architecture progressively separates high-frequency components within low-frequency features, ensuring a stable structure for motion magnification.
3. **Sparse High/Low-pass Filters**: These filters are designed to mitigate the degradation of high-frequency details and low-frequency structures caused by noise, enhancing the integrity of the motion field.
4. **Sparse Frequency Mixer**: This component promotes seamless recoupling of high-frequency details and magnified low-frequency features, reducing ringing artifacts.
5. **Contrastive Regularization**: A novel loss function is introduced to strengthen the model's ability to discriminate irrelevant features, reducing undesired motion magnification.
Experiments on both real-world and synthetic datasets demonstrate that FD4MM outperforms state-of-the-art methods in terms of magnification quality, with fewer FLOPs and faster inference speed. The paper also includes detailed ablation studies and qualitative comparisons to validate the effectiveness of each component and the overall framework.