MENTOR: Multi-level Self-supervised Learning for Multimodal Recommendation

MENTOR: Multi-level Self-supervised Learning for Multimodal Recommendation

2024 | Jinfeng Xu, Zheyu Chen, Shuo Yang, Jinze Li, Hewei Wang, Edith C.-H. Ngai
The paper introduces MENTOR, a novel self-supervised learning framework for multimodal recommendation. MENTOR addresses the challenges of data sparsity and label sparsity in recommendation systems by leveraging multimodal information. It enhances specific features of each modality using graph convolutional networks (GCNs) and fuses visual and textual modalities. The model then enhances item representations via an item semantic graph for all modalities. Two key components are introduced: a multilevel cross-modal alignment task and a general feature enhancement task. The multilevel cross-modal alignment task aligns different modalities while maintaining historical interaction information, while the general feature enhancement task improves model robustness through feature masking and graph perturbation. Extensive experiments on three public datasets demonstrate the effectiveness of MENTOR, showing significant improvements over state-of-the-art methods. The code for MENTOR is publicly available.The paper introduces MENTOR, a novel self-supervised learning framework for multimodal recommendation. MENTOR addresses the challenges of data sparsity and label sparsity in recommendation systems by leveraging multimodal information. It enhances specific features of each modality using graph convolutional networks (GCNs) and fuses visual and textual modalities. The model then enhances item representations via an item semantic graph for all modalities. Two key components are introduced: a multilevel cross-modal alignment task and a general feature enhancement task. The multilevel cross-modal alignment task aligns different modalities while maintaining historical interaction information, while the general feature enhancement task improves model robustness through feature masking and graph perturbation. Extensive experiments on three public datasets demonstrate the effectiveness of MENTOR, showing significant improvements over state-of-the-art methods. The code for MENTOR is publicly available.
Reach us at info@study.space
[slides and audio] MENTOR%3A Multi-level Self-supervised Learning for Multimodal Recommendation