Fréchet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videos

Fréchet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videos

2024 | Jiahe Liu, Youran Qu, Qi Yan, Xiaohui Zeng, Lele Wang, Renjie Liao
Fréchet Video Motion Distance (FVMD) is a novel metric designed to evaluate motion consistency in video generation. Unlike existing metrics that focus on visual quality, FVMD specifically addresses the temporal coherence and motion patterns in videos. The metric is based on extracting motion features from key point tracking, computing velocity and acceleration fields, and then measuring the similarity between these features using the Fréchet distance. The method involves tracking key points in videos using a pre-trained model, calculating velocity and acceleration fields, and then deriving motion features from these fields. These motion features are then used to compute the Fréchet distance between generated and ground-truth videos, providing a measure of motion consistency. The proposed FVMD metric was validated through sensitivity analysis and large-scale human studies. The results showed that FVMD effectively captures temporal noise and aligns better with human perceptions of video quality compared to existing metrics. Additionally, the motion features derived from FVMD consistently improve the performance of Video Quality Assessment (VQA) models, indicating their potential for unary video quality evaluation. The metric was also tested on various video generation models, demonstrating its effectiveness in distinguishing high-quality videos from low-quality ones. The FVMD metric was implemented with a focus on efficiency, with the majority of the runtime consumed by the video tracking stage. The method was evaluated on different video datasets, including TikTok and BAIR, and showed strong performance in distinguishing temporal inconsistencies. The results of the human studies indicated that FVMD outperforms existing metrics in aligning with human judgment, particularly in scenarios where other metrics fail to evaluate video quality accurately. Overall, FVMD provides a more comprehensive assessment of video quality compared to existing metrics, making it a valuable tool for evaluating motion consistency in video generation.Fréchet Video Motion Distance (FVMD) is a novel metric designed to evaluate motion consistency in video generation. Unlike existing metrics that focus on visual quality, FVMD specifically addresses the temporal coherence and motion patterns in videos. The metric is based on extracting motion features from key point tracking, computing velocity and acceleration fields, and then measuring the similarity between these features using the Fréchet distance. The method involves tracking key points in videos using a pre-trained model, calculating velocity and acceleration fields, and then deriving motion features from these fields. These motion features are then used to compute the Fréchet distance between generated and ground-truth videos, providing a measure of motion consistency. The proposed FVMD metric was validated through sensitivity analysis and large-scale human studies. The results showed that FVMD effectively captures temporal noise and aligns better with human perceptions of video quality compared to existing metrics. Additionally, the motion features derived from FVMD consistently improve the performance of Video Quality Assessment (VQA) models, indicating their potential for unary video quality evaluation. The metric was also tested on various video generation models, demonstrating its effectiveness in distinguishing high-quality videos from low-quality ones. The FVMD metric was implemented with a focus on efficiency, with the majority of the runtime consumed by the video tracking stage. The method was evaluated on different video datasets, including TikTok and BAIR, and showed strong performance in distinguishing temporal inconsistencies. The results of the human studies indicated that FVMD outperforms existing metrics in aligning with human judgment, particularly in scenarios where other metrics fail to evaluate video quality accurately. Overall, FVMD provides a more comprehensive assessment of video quality compared to existing metrics, making it a valuable tool for evaluating motion consistency in video generation.
Reach us at info@study.space