OAG-Bench: A Human-Curated Benchmark for Academic Graph Mining

OAG-Bench: A Human-Curated Benchmark for Academic Graph Mining

August 25–29, 2024, Barcelona, Spain | Fanjin Zhang, Shijie Shi, Yifan Zhu, Bo Chen, Yukuo Cen, Jifan Yu, Yelin Chen, Lulu Wang, Qingfei Zhao, Yuqing Cheng, Tianyi Han, Yuwei An, Dan Zhang, Weng Lam Tam, Kun Cao, Yunhe Pang, Xinyu Guan, Huihui Yuan, Jian Song, Xiaoyan Li, Yuxiao Dong, Jie Tang
OAG-Bench is a comprehensive, multi-aspect, and fine-grained human-curated benchmark for academic graph mining, built on the Open Academic Graph (OAG). The benchmark covers 10 tasks, 20 datasets, 70+ baseline methods, and 120+ experimental results. It aims to address the limitations of existing academic graphs and benchmarks by providing detailed annotations, data preprocessing codes, algorithm implementations, and standardized evaluation protocols. OAG-Bench includes tasks such as author name disambiguation, scholar profiling, paper source tracing, and academic question answering. The paper also introduces the Open Academic Graph Challenge (OAG-Challenge) to encourage community participation and sharing. The benchmark is designed to facilitate the development and evaluation of advanced algorithms in academic graph mining, making it a valuable resource for researchers and practitioners.OAG-Bench is a comprehensive, multi-aspect, and fine-grained human-curated benchmark for academic graph mining, built on the Open Academic Graph (OAG). The benchmark covers 10 tasks, 20 datasets, 70+ baseline methods, and 120+ experimental results. It aims to address the limitations of existing academic graphs and benchmarks by providing detailed annotations, data preprocessing codes, algorithm implementations, and standardized evaluation protocols. OAG-Bench includes tasks such as author name disambiguation, scholar profiling, paper source tracing, and academic question answering. The paper also introduces the Open Academic Graph Challenge (OAG-Challenge) to encourage community participation and sharing. The benchmark is designed to facilitate the development and evaluation of advanced algorithms in academic graph mining, making it a valuable resource for researchers and practitioners.
Reach us at info@study.space
[slides and audio] OAG-Bench%3A A Human-Curated Benchmark for Academic Graph Mining