The paper introduces the Receptive Field Block (RFB) module, which enhances the feature representation of lightweight CNN models to improve the accuracy and speed of object detection. Inspired by the structure of Receptive Fields (RFs) in human visual systems, the RFB module uses multi-branch pooling with varying kernels to simulate different RF sizes and dilated convolution layers to control their eccentricities. The RFB module is integrated into the Single Shot Detector (SSD) to form the RFB Net detector, achieving state-of-the-art performance on the Pascal VOC and MS COCO datasets while maintaining real-time processing speed. The effectiveness of the RFB module is demonstrated through experiments, showing significant improvements over existing detectors, including SSD, YOLO, and advanced one-stage detectors like RetinaNet. The RFB module is also shown to be effective when combined with other lightweight backbones, such as MobileNet, and can be trained from scratch, making it suitable for low-end devices.The paper introduces the Receptive Field Block (RFB) module, which enhances the feature representation of lightweight CNN models to improve the accuracy and speed of object detection. Inspired by the structure of Receptive Fields (RFs) in human visual systems, the RFB module uses multi-branch pooling with varying kernels to simulate different RF sizes and dilated convolution layers to control their eccentricities. The RFB module is integrated into the Single Shot Detector (SSD) to form the RFB Net detector, achieving state-of-the-art performance on the Pascal VOC and MS COCO datasets while maintaining real-time processing speed. The effectiveness of the RFB module is demonstrated through experiments, showing significant improvements over existing detectors, including SSD, YOLO, and advanced one-stage detectors like RetinaNet. The RFB module is also shown to be effective when combined with other lightweight backbones, such as MobileNet, and can be trained from scratch, making it suitable for low-end devices.