Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation Models

Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation Models

28 May 2024 | Xingtong Yu, Chang Zhou, Yuan Fang, Xinming Zhang
MDGPT is a text-free multi-domain graph pre-training and adaptation framework designed to learn from diverse graph domains and adapt to both seen and unseen domains. The framework addresses the challenges of aligning multi-domain graphs and adapting pre-trained knowledge to downstream tasks. The key contributions include: (1) proposing MDGPT, a text-free multi-domain pre-training and prompting framework for graphs; (2) designing domain tokens to align node features during pre-training; (3) introducing dual prompts for downstream adaptation, including a unifying prompt and a mixing prompt; and (4) conducting extensive experiments on five benchmark datasets, demonstrating the superior performance of MDGPT compared to state-of-the-art approaches. The pre-training process involves aligning feature dimensions across multiple source domains and unifying semantic spaces through domain tokens. The downstream adaptation process uses dual prompts to align the target domain with pre-trained knowledge, enabling effective transfer of multi-domain knowledge to downstream tasks. MDGPT outperforms existing methods by up to 37.9% in performance on six public datasets, showing its effectiveness in multi-domain pre-training and downstream adaptation. The framework is particularly suitable for few-shot learning scenarios, where only a limited number of labeled examples are available for downstream tasks.MDGPT is a text-free multi-domain graph pre-training and adaptation framework designed to learn from diverse graph domains and adapt to both seen and unseen domains. The framework addresses the challenges of aligning multi-domain graphs and adapting pre-trained knowledge to downstream tasks. The key contributions include: (1) proposing MDGPT, a text-free multi-domain pre-training and prompting framework for graphs; (2) designing domain tokens to align node features during pre-training; (3) introducing dual prompts for downstream adaptation, including a unifying prompt and a mixing prompt; and (4) conducting extensive experiments on five benchmark datasets, demonstrating the superior performance of MDGPT compared to state-of-the-art approaches. The pre-training process involves aligning feature dimensions across multiple source domains and unifying semantic spaces through domain tokens. The downstream adaptation process uses dual prompts to align the target domain with pre-trained knowledge, enabling effective transfer of multi-domain knowledge to downstream tasks. MDGPT outperforms existing methods by up to 37.9% in performance on six public datasets, showing its effectiveness in multi-domain pre-training and downstream adaptation. The framework is particularly suitable for few-shot learning scenarios, where only a limited number of labeled examples are available for downstream tasks.
Reach us at info@study.space
[slides] Text-Free Multi-domain Graph Pre-training%3A Toward Graph Foundation Models | StudySpace