July 14–18, 2024, Washington, DC, USA | Zijian Zhang, Shuchang Liu, Jiaoo Yu, Qingpeng Cai, Xiangyu Zhao, Chunxu Zhang, Ziru Liu, Qidong Liu, Hongwei Zhao, Lantao Hu, Peng Jiang, Kun Gai
**M³oE: Multi-Domain Multi-Task Mixture-of-Experts Recommendation Framework**
**Authors:**
- Zijian Zhang (Jilin University)
- Shuchang Liu (Kuaishou Technology)
- Jiaojia Yu (Kuaishou Technology)
- Qingpeng Cai (Kuaishou Technology)
- Xiangyu Zhao (City University of Hong Kong)
- Chunxu Zhang (Jilin University)
- Ziru Liu (City University of Hong Kong)
- Qidong Liu (Xi'an Jiaotong University)
- Hongwei Zhao (Jilin University)
- Lantao Hu (Kuaishou Technology)
- Peng Jiang (Kuaishou Technology)
- Kun Gai (Unaffiliated)
**Abstract:**
Multi-domain and multi-task recommendation have shown effectiveness in leveraging common information from different domains and objectives for comprehensive user modeling. However, practical recommendation often faces multiple domains and tasks simultaneously, which current methods struggle to address. To tackle this, we introduce M³oE, an adaptive Multi-domain Multi-task Mixture-of-Experts recommendation framework. M³oE integrates multi-domain information, maps knowledge across domains and tasks, and optimizes multiple objectives. It uses three mixture-of-experts modules to learn common, domain-aspect, and task-aspect user preferences, addressing complex dependencies in a disentangled manner. A two-level fusion mechanism controls feature extraction and fusion across diverse domains and tasks. AutoML is applied to enhance adaptability by dynamically optimizing the framework's structure. Extensive experiments on two benchmark datasets demonstrate M³oE's superior performance compared to diverse baselines, highlighting its effectiveness in solving multi-domain multi-task recommendation self-adaptively.
**Keywords:**
Recommender System; Multi-Domain; Multi-Task
**CCS Concepts:**
- Information systems → Recommender systems.
**Introduction:**
The paper addresses the challenges of multi-domain and multi-task recommendation, where users may exhibit similar tastes across domains and interact with items in various ways. Current methods often struggle to effectively handle both scenarios simultaneously. M³oE aims to bridge this gap by integrating multi-domain information, mapping knowledge across domains and tasks, and optimizing multiple objectives. The framework consists of a domain representation extraction layer, a multi-view expert learning layer, and an MDMT objective prediction layer. It employs three types of experts—shared, domain-specific, and task-specific—to capture common, domain-aspect, and task-aspect user preferences, respectively. A two-level fusion mechanism controls the integration of these experts, and AutoML is used to optimize the fusion weights adaptively. Extensive experiments on MovieLens and KuaRand-Pure datasets show M³oE's superior performance, demonstrating its effectiveness in multi-domain multi-task recommendation.
**Methodology:**
The paper defines the multi-domain multi-task recommendation problem and outlines the M³oE framework. It details**M³oE: Multi-Domain Multi-Task Mixture-of-Experts Recommendation Framework**
**Authors:**
- Zijian Zhang (Jilin University)
- Shuchang Liu (Kuaishou Technology)
- Jiaojia Yu (Kuaishou Technology)
- Qingpeng Cai (Kuaishou Technology)
- Xiangyu Zhao (City University of Hong Kong)
- Chunxu Zhang (Jilin University)
- Ziru Liu (City University of Hong Kong)
- Qidong Liu (Xi'an Jiaotong University)
- Hongwei Zhao (Jilin University)
- Lantao Hu (Kuaishou Technology)
- Peng Jiang (Kuaishou Technology)
- Kun Gai (Unaffiliated)
**Abstract:**
Multi-domain and multi-task recommendation have shown effectiveness in leveraging common information from different domains and objectives for comprehensive user modeling. However, practical recommendation often faces multiple domains and tasks simultaneously, which current methods struggle to address. To tackle this, we introduce M³oE, an adaptive Multi-domain Multi-task Mixture-of-Experts recommendation framework. M³oE integrates multi-domain information, maps knowledge across domains and tasks, and optimizes multiple objectives. It uses three mixture-of-experts modules to learn common, domain-aspect, and task-aspect user preferences, addressing complex dependencies in a disentangled manner. A two-level fusion mechanism controls feature extraction and fusion across diverse domains and tasks. AutoML is applied to enhance adaptability by dynamically optimizing the framework's structure. Extensive experiments on two benchmark datasets demonstrate M³oE's superior performance compared to diverse baselines, highlighting its effectiveness in solving multi-domain multi-task recommendation self-adaptively.
**Keywords:**
Recommender System; Multi-Domain; Multi-Task
**CCS Concepts:**
- Information systems → Recommender systems.
**Introduction:**
The paper addresses the challenges of multi-domain and multi-task recommendation, where users may exhibit similar tastes across domains and interact with items in various ways. Current methods often struggle to effectively handle both scenarios simultaneously. M³oE aims to bridge this gap by integrating multi-domain information, mapping knowledge across domains and tasks, and optimizing multiple objectives. The framework consists of a domain representation extraction layer, a multi-view expert learning layer, and an MDMT objective prediction layer. It employs three types of experts—shared, domain-specific, and task-specific—to capture common, domain-aspect, and task-aspect user preferences, respectively. A two-level fusion mechanism controls the integration of these experts, and AutoML is used to optimize the fusion weights adaptively. Extensive experiments on MovieLens and KuaRand-Pure datasets show M³oE's superior performance, demonstrating its effectiveness in multi-domain multi-task recommendation.
**Methodology:**
The paper defines the multi-domain multi-task recommendation problem and outlines the M³oE framework. It details