8 May 2024 | Yue Hu, Juntong Peng, Sifei Liu, Junhao Ge, Si Liu, Siheng Chen
CodeFilling is a novel communication-efficient collaborative 3D detection system that improves the perception-communication trade-off. It uses a codebook-based message representation to transmit integer codes instead of high-dimensional feature maps, reducing communication costs. Additionally, it employs an information-filling-driven message selection strategy to optimize local messages, ensuring that each agent's information demand is met without information overflow. CodeFilling is evaluated on real-world and simulation datasets, showing significant improvements in communication efficiency compared to previous methods. It outperforms Where2comm by 1,333/1,206 times in communication volume on DAIR-V2X and OPV2VH+ datasets. CodeFilling is effective in both homogeneous and heterogeneous collaboration settings. The system includes four key modules: a single-agent detector, information-filling-driven message selection, codebook-based message representation, and message decoding and fusion. The codebook-based representation allows for efficient transmission of integer code indices, while the information-filling strategy ensures that only necessary information is shared. CodeFilling achieves superior performance in both real-world and simulation scenarios, demonstrating robustness to pose errors and communication latency. The system is also efficient in terms of inference speed and maintains a superior perception-communication trade-off across various communication conditions.CodeFilling is a novel communication-efficient collaborative 3D detection system that improves the perception-communication trade-off. It uses a codebook-based message representation to transmit integer codes instead of high-dimensional feature maps, reducing communication costs. Additionally, it employs an information-filling-driven message selection strategy to optimize local messages, ensuring that each agent's information demand is met without information overflow. CodeFilling is evaluated on real-world and simulation datasets, showing significant improvements in communication efficiency compared to previous methods. It outperforms Where2comm by 1,333/1,206 times in communication volume on DAIR-V2X and OPV2VH+ datasets. CodeFilling is effective in both homogeneous and heterogeneous collaboration settings. The system includes four key modules: a single-agent detector, information-filling-driven message selection, codebook-based message representation, and message decoding and fusion. The codebook-based representation allows for efficient transmission of integer code indices, while the information-filling strategy ensures that only necessary information is shared. CodeFilling achieves superior performance in both real-world and simulation scenarios, demonstrating robustness to pose errors and communication latency. The system is also efficient in terms of inference speed and maintains a superior perception-communication trade-off across various communication conditions.