5 May 2024 | Qingyang Zhang, Yake Wei, Zongbo Han, Huazhu Fu, Xi Peng, Cheng Deng, Qinghua Hu, Cai Xu, Jie Wen, Di Hu, Changqing Zhang
This paper presents a comprehensive survey on multimodal fusion under low-quality data settings. It identifies four main challenges: (1) noisy multimodal data with heterogeneous noises, (2) incomplete multimodal data with missing modalities, (3) imbalanced multimodal data with varying qualities, and (4) quality-varying multimodal data with dynamic quality changes. The paper discusses recent advances in addressing these challenges, including noise reduction techniques, imputation methods for incomplete data, balanced fusion approaches, and dynamic fusion strategies. It also highlights open problems and future research directions in this area. The survey covers various methods for learning from noisy multimodal data, including modal-specific and cross-modal noise reduction techniques. For incomplete multimodal data, the paper discusses imputation-based and imputation-free approaches, including matrix factorization, graph learning, kernel learning, and deep learning methods. The survey emphasizes the importance of handling noise and missing data in multimodal fusion to achieve robust and reliable results in real-world applications.This paper presents a comprehensive survey on multimodal fusion under low-quality data settings. It identifies four main challenges: (1) noisy multimodal data with heterogeneous noises, (2) incomplete multimodal data with missing modalities, (3) imbalanced multimodal data with varying qualities, and (4) quality-varying multimodal data with dynamic quality changes. The paper discusses recent advances in addressing these challenges, including noise reduction techniques, imputation methods for incomplete data, balanced fusion approaches, and dynamic fusion strategies. It also highlights open problems and future research directions in this area. The survey covers various methods for learning from noisy multimodal data, including modal-specific and cross-modal noise reduction techniques. For incomplete multimodal data, the paper discusses imputation-based and imputation-free approaches, including matrix factorization, graph learning, kernel learning, and deep learning methods. The survey emphasizes the importance of handling noise and missing data in multimodal fusion to achieve robust and reliable results in real-world applications.