April 14–20, 2024, Lisbon, Portugal | Xinyu Gao, Zhijie Wang, Yang Feng, Lei Ma, Zhenyu Chen, Baowen Xu
MultiTest is an automated testing method designed for multi-sensor fusion (MSF) perception systems, addressing the limitations of existing testing methods that primarily focus on single-sensor systems. It employs a physical-aware approach to synthesize realistic multi-modal object instances and insert them into critical positions of background images and point clouds. A fitness metric guides the test generation process, enhancing the effectiveness and efficiency of testing. Extensive experiments with five state-of-the-art (SOTA) perception systems demonstrate that MultiTest can generate realistic and modality-consistent test data, effectively detect various faults, and improve system performance through retraining. The tool and synthesized testing dataset are publicly available, making it a valuable resource for the AI-enabled MSF community.MultiTest is an automated testing method designed for multi-sensor fusion (MSF) perception systems, addressing the limitations of existing testing methods that primarily focus on single-sensor systems. It employs a physical-aware approach to synthesize realistic multi-modal object instances and insert them into critical positions of background images and point clouds. A fitness metric guides the test generation process, enhancing the effectiveness and efficiency of testing. Extensive experiments with five state-of-the-art (SOTA) perception systems demonstrate that MultiTest can generate realistic and modality-consistent test data, effectively detect various faults, and improve system performance through retraining. The tool and synthesized testing dataset are publicly available, making it a valuable resource for the AI-enabled MSF community.