STAR: A First-Ever Dataset and A Large-Scale Benchmark for Scene Graph Generation in Large-Size Satellite Imagery

STAR: A First-Ever Dataset and A Large-Scale Benchmark for Scene Graph Generation in Large-Size Satellite Imagery

3 Jul 2024 | Yansheng Li, Senior Member, IEEE, Linlin Wang, Tingzhu Wang, Xue Yang, Junwei Luo, Qi Wang, Senior Member, IEEE,Youming Deng, Wenbin Wang, Xian Sun, Senior Member, IEEE, Haifeng Li, Member, IEEE, Bo Dang, Yongjun Zhang, Member, IEEE, Yi Yu, and Junchi Yan, Senior Member, IEEE
The paper introduces STAR, a novel dataset and benchmark for scene graph generation (SGG) in large-size very-high-resolution (VHR) satellite imagery. The dataset, named STAR, contains over 210,000 objects and 400,000 triplets, spanning 11 categories of complex geospatial scenarios. It addresses the challenges of object detection, pair pruning, and relationship prediction in large-size VHR SAI. The authors propose a context-aware cascade cognition (CAC) framework, which includes a holistic multi-class object detection network (HOD-Net), a pair proposal generation (PPG) network, and a relationship prediction network with context-aware messaging (RPCM). The CAC framework aims to enhance the cognitive understanding of SAI by integrating multi-scale context, pruning high-value pairs, and inferring relationships based on context. The paper also releases an open-source toolkit with about 30 object detection methods and 10 SGG methods, providing a comprehensive resource for the community. The experimental results on the STAR dataset demonstrate the effectiveness of the proposed framework and the dataset's value for future research in SGG for large-size VHR SAI.The paper introduces STAR, a novel dataset and benchmark for scene graph generation (SGG) in large-size very-high-resolution (VHR) satellite imagery. The dataset, named STAR, contains over 210,000 objects and 400,000 triplets, spanning 11 categories of complex geospatial scenarios. It addresses the challenges of object detection, pair pruning, and relationship prediction in large-size VHR SAI. The authors propose a context-aware cascade cognition (CAC) framework, which includes a holistic multi-class object detection network (HOD-Net), a pair proposal generation (PPG) network, and a relationship prediction network with context-aware messaging (RPCM). The CAC framework aims to enhance the cognitive understanding of SAI by integrating multi-scale context, pruning high-value pairs, and inferring relationships based on context. The paper also releases an open-source toolkit with about 30 object detection methods and 10 SGG methods, providing a comprehensive resource for the community. The experimental results on the STAR dataset demonstrate the effectiveness of the proposed framework and the dataset's value for future research in SGG for large-size VHR SAI.
Reach us at info@study.space