August 25-29, 2024 | Yuhan Li, Peisong Wang, Zhixun Li, Jeffrey Xu Yu, Jia Li
ZEROG is a novel framework for cross-dataset zero-shot transferability in graphs. The paper addresses the challenges of zero-shot transfer learning in graph learning, including feature misalignment, mismatched label spaces, and negative transfer. ZEROG leverages a pre-trained language model to encode both node attributes and class semantics, ensuring consistent feature dimensions across datasets. It also proposes a prompt-based subgraph sampling module that enriches the semantic information and structure information of extracted subgraphs. Additionally, a lightweight fine-tuning strategy is adopted to reduce the risk of overfitting and maintain the zero-shot learning efficacy of the language model. The results show that ZEROG achieves significant cross-dataset zero-shot transferability, demonstrating its effectiveness in achieving zero-shot learning on various graph datasets. The paper also discusses the challenges of zero-shot learning in graph learning, including the limitations of traditional GNNs and the shortcomings of large language models in cross-dataset zero-shot node classification. The methodology of ZEROG includes a unified graph representation, prompt-based subgraph sampling, and upstream pre-training. The experiments show that ZEROG outperforms existing methods in cross-dataset zero-shot transferability, with significant improvements in accuracy on benchmark datasets. The paper also discusses the efficiency of ZEROG, showing that it requires fewer parameters and training time compared to other methods. Overall, ZEROG provides a promising approach for cross-dataset zero-shot transferability in graphs, with potential applications in graph foundation models.ZEROG is a novel framework for cross-dataset zero-shot transferability in graphs. The paper addresses the challenges of zero-shot transfer learning in graph learning, including feature misalignment, mismatched label spaces, and negative transfer. ZEROG leverages a pre-trained language model to encode both node attributes and class semantics, ensuring consistent feature dimensions across datasets. It also proposes a prompt-based subgraph sampling module that enriches the semantic information and structure information of extracted subgraphs. Additionally, a lightweight fine-tuning strategy is adopted to reduce the risk of overfitting and maintain the zero-shot learning efficacy of the language model. The results show that ZEROG achieves significant cross-dataset zero-shot transferability, demonstrating its effectiveness in achieving zero-shot learning on various graph datasets. The paper also discusses the challenges of zero-shot learning in graph learning, including the limitations of traditional GNNs and the shortcomings of large language models in cross-dataset zero-shot node classification. The methodology of ZEROG includes a unified graph representation, prompt-based subgraph sampling, and upstream pre-training. The experiments show that ZEROG outperforms existing methods in cross-dataset zero-shot transferability, with significant improvements in accuracy on benchmark datasets. The paper also discusses the efficiency of ZEROG, showing that it requires fewer parameters and training time compared to other methods. Overall, ZEROG provides a promising approach for cross-dataset zero-shot transferability in graphs, with potential applications in graph foundation models.