This paper addresses the challenge of obstacle intrusion in railway traffic safety, particularly in complex weather conditions. It proposes a multi-modal contrastive learning strategy named DHT-CL to enhance the performance of LiDAR point cloud 3D semantic segmentation (3DSS) for rail-obstacle detection. The DHT-CL strategy integrates camera and LiDAR sensors to improve robustness under rainy and snowy conditions. The Dual-Helix Transformer (DHT) module is designed to extract deeper cross-modal information through a neighborhood attention mechanism, while an adaptive cross-modal discrimination loss is constructed to adapt to the anomaly identification task. Experimental results on a complex weather railway dataset show that DHT-CL achieves an mIoU of 87.38%, outperforming other high-performance models from the autonomous driving dataset, SemanticKITTI. Qualitative results demonstrate higher accuracy in clear weather and reduced false alarms in rainy and snowy weather. The proposed method improves the generalization and robustness of deep learning-based methods in complex weather conditions.This paper addresses the challenge of obstacle intrusion in railway traffic safety, particularly in complex weather conditions. It proposes a multi-modal contrastive learning strategy named DHT-CL to enhance the performance of LiDAR point cloud 3D semantic segmentation (3DSS) for rail-obstacle detection. The DHT-CL strategy integrates camera and LiDAR sensors to improve robustness under rainy and snowy conditions. The Dual-Helix Transformer (DHT) module is designed to extract deeper cross-modal information through a neighborhood attention mechanism, while an adaptive cross-modal discrimination loss is constructed to adapt to the anomaly identification task. Experimental results on a complex weather railway dataset show that DHT-CL achieves an mIoU of 87.38%, outperforming other high-performance models from the autonomous driving dataset, SemanticKITTI. Qualitative results demonstrate higher accuracy in clear weather and reduced false alarms in rainy and snowy weather. The proposed method improves the generalization and robustness of deep learning-based methods in complex weather conditions.