28 Mar 2024 | Lingjun Zhao*, Jingyu Song*, Katherine A. Skinner
The paper introduces CRKD (Camera-Radar Knowledge Distillation), a novel framework that enhances Camera-Radar (CR) 3D object detection through cross-modality knowledge distillation. CRKD aims to bridge the performance gap between CR and LiDAR-Camera (LC) fusion, which is the top-performing sensor configuration for autonomous driving but is costly. The proposed framework leverages the Bird’s-Eye-View (BEV) representation as a shared feature space to enable effective knowledge distillation. Four distillation losses are designed to address the unique challenges of cross-modality distillation, including Cross-Stage Radar Distillation (CSRD), Mask-Scaling Feature Distillation (MSFD), Relation Distillation (RelD), and Response Distillation (RespD). Extensive evaluations on the nuScenes dataset demonstrate that CRKD improves the mAP and NDS of the student detector by 3.5% and 3.2%, respectively, compared to existing baselines. The project page for CRKD is available at <https://song-jingyu.github.io/CRKD>.The paper introduces CRKD (Camera-Radar Knowledge Distillation), a novel framework that enhances Camera-Radar (CR) 3D object detection through cross-modality knowledge distillation. CRKD aims to bridge the performance gap between CR and LiDAR-Camera (LC) fusion, which is the top-performing sensor configuration for autonomous driving but is costly. The proposed framework leverages the Bird’s-Eye-View (BEV) representation as a shared feature space to enable effective knowledge distillation. Four distillation losses are designed to address the unique challenges of cross-modality distillation, including Cross-Stage Radar Distillation (CSRD), Mask-Scaling Feature Distillation (MSFD), Relation Distillation (RelD), and Response Distillation (RespD). Extensive evaluations on the nuScenes dataset demonstrate that CRKD improves the mAP and NDS of the student detector by 3.5% and 3.2%, respectively, compared to existing baselines. The project page for CRKD is available at <https://song-jingyu.github.io/CRKD>.