24 Mar 2024 | Fei Wang, Dan Guo, Kun Li, Zhun Zhong, Meng Wang
This paper proposes FD4MM, a novel method for video motion magnification (VMM) based on frequency decoupling with a multi-level isomorphic architecture. VMM aims to reveal subtle motion information in videos. Existing methods often struggle with noise and degradation, especially for high-frequency details and subtle motions. FD4MM addresses these challenges by separating high- and low-frequency components using frequency decoupling. The low-frequency components are used to model the stable motion field, while high-frequency components are used to preserve spatial details.
FD4MM introduces sparse high-pass and low-pass filters to enhance the integrity of details and motion structures, and a sparse frequency mixer to promote seamless recoupling. Additionally, a contrastive regularization is proposed to strengthen the model's ability to discriminate irrelevant features, reducing undesired motion magnification.
The method is evaluated on both real-world and synthetic datasets, showing that FD4MM outperforms state-of-the-art methods in terms of magnification quality, with a 1.63× reduction in FLOPs and a 1.68× increase in inference speed. The multi-level isomorphic architecture enables the model to capture multi-level high-frequency details and a stable low-frequency structure. The sparse filters and mixer effectively handle noise and degradation, ensuring accurate and robust motion magnification. The results demonstrate that FD4MM achieves superior performance in preserving spatial consistency and reducing artifacts, making it a promising solution for future research in motion magnification.This paper proposes FD4MM, a novel method for video motion magnification (VMM) based on frequency decoupling with a multi-level isomorphic architecture. VMM aims to reveal subtle motion information in videos. Existing methods often struggle with noise and degradation, especially for high-frequency details and subtle motions. FD4MM addresses these challenges by separating high- and low-frequency components using frequency decoupling. The low-frequency components are used to model the stable motion field, while high-frequency components are used to preserve spatial details.
FD4MM introduces sparse high-pass and low-pass filters to enhance the integrity of details and motion structures, and a sparse frequency mixer to promote seamless recoupling. Additionally, a contrastive regularization is proposed to strengthen the model's ability to discriminate irrelevant features, reducing undesired motion magnification.
The method is evaluated on both real-world and synthetic datasets, showing that FD4MM outperforms state-of-the-art methods in terms of magnification quality, with a 1.63× reduction in FLOPs and a 1.68× increase in inference speed. The multi-level isomorphic architecture enables the model to capture multi-level high-frequency details and a stable low-frequency structure. The sparse filters and mixer effectively handle noise and degradation, ensuring accurate and robust motion magnification. The results demonstrate that FD4MM achieves superior performance in preserving spatial consistency and reducing artifacts, making it a promising solution for future research in motion magnification.