HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model

HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model

17 Jun 2024 | Di Wang*, Meiqi Hu*, Yao Jin*, Yuchun Miao*, Jiaqi Yang*, Yichu Xu*, Xiaolei Qin*, Jiaqi Ma*, Lingyu Sun*, Chenxing Li*, Chuan Fu, Hongruxuan Chen, Chengxi Han†, Naoto Yokoya, Member, IEEE, Jing Zhang†, Senior Member, IEEE, Mingjiang Xu, Lin Liu, Lefei Zhang, Senior Member, IEEE, Chen Wu†, Member, IEEE, Bo Du†, Senior Member, IEEE, Dacheng Tao, Fellow, IEEE and Liangpei Zhang†, Fellow, IEEE
HyperSIGMA is a foundation model designed for the interpretation of hyperspectral images (HSIs), addressing the challenges of high dimensionality, data redundancy, and spatial variability in HSIs. The model integrates spatial and spectral features using a spectral enhancement module and introduces a novel sparse sampling attention (SSA) mechanism to effectively learn diverse contextual features. HyperSIGMA is pre-trained on the HyperGlobal-450K dataset, which contains about 450,000 hyperspectral images, significantly surpassing existing datasets in scale. Extensive experiments demonstrate HyperSIGMA's versatility and superior representational capability across various high-level and low-level HSI tasks, including image classification, target detection, anomaly detection, change detection, spectral unmixing, image denoising, and super-resolution. HyperSIGMA also shows significant advantages in scalability, robustness, cross-modal transferring capability, and real-world applicability. The code and models will be released to promote further research and applications in hyperspectral remote sensing.HyperSIGMA is a foundation model designed for the interpretation of hyperspectral images (HSIs), addressing the challenges of high dimensionality, data redundancy, and spatial variability in HSIs. The model integrates spatial and spectral features using a spectral enhancement module and introduces a novel sparse sampling attention (SSA) mechanism to effectively learn diverse contextual features. HyperSIGMA is pre-trained on the HyperGlobal-450K dataset, which contains about 450,000 hyperspectral images, significantly surpassing existing datasets in scale. Extensive experiments demonstrate HyperSIGMA's versatility and superior representational capability across various high-level and low-level HSI tasks, including image classification, target detection, anomaly detection, change detection, spectral unmixing, image denoising, and super-resolution. HyperSIGMA also shows significant advantages in scalability, robustness, cross-modal transferring capability, and real-world applicability. The code and models will be released to promote further research and applications in hyperspectral remote sensing.
Reach us at info@study.space
Understanding HyperSIGMA%3A Hyperspectral Intelligence Comprehension Foundation Model