30 Oct 2018 | Golnaz Ghiassi, Tsung-Yi Lin, Quoc V. Le
DropBlock is a regularization method designed for convolutional neural networks (CNNs). Unlike dropout, which randomly drops units in a fully connected layer, DropBlock drops contiguous regions of a feature map, effectively removing spatially correlated information. This method is particularly effective for convolutional layers where features are spatially correlated, making it more robust against overfitting compared to traditional dropout.
The paper introduces DropBlock and compares its performance with dropout and other regularization techniques such as SpatialDropout, DropPath, and Cutout. Experiments on the ImageNet dataset show that DropBlock improves accuracy for the ResNet-50 architecture by 1.6%, achieving 78.13% accuracy. On the COCO dataset, DropBlock enhances the Average Precision of RetinaNet from 36.8% to 38.4%.
DropBlock is applied to both convolution layers and skip connections in residual networks, and the results show that applying DropBlock to skip connections further improves accuracy. The paper also demonstrates that gradually increasing the number of dropped units during training leads to better accuracy and robustness to hyperparameter choices.
The effectiveness of DropBlock is further analyzed through visualizations and experiments, showing that models trained with DropBlock learn more spatially distributed representations and are more robust to changes in hyperparameters. Overall, DropBlock is a powerful tool for regularizing CNNs, especially in convolutional layers, and shows significant improvements over traditional dropout methods.DropBlock is a regularization method designed for convolutional neural networks (CNNs). Unlike dropout, which randomly drops units in a fully connected layer, DropBlock drops contiguous regions of a feature map, effectively removing spatially correlated information. This method is particularly effective for convolutional layers where features are spatially correlated, making it more robust against overfitting compared to traditional dropout.
The paper introduces DropBlock and compares its performance with dropout and other regularization techniques such as SpatialDropout, DropPath, and Cutout. Experiments on the ImageNet dataset show that DropBlock improves accuracy for the ResNet-50 architecture by 1.6%, achieving 78.13% accuracy. On the COCO dataset, DropBlock enhances the Average Precision of RetinaNet from 36.8% to 38.4%.
DropBlock is applied to both convolution layers and skip connections in residual networks, and the results show that applying DropBlock to skip connections further improves accuracy. The paper also demonstrates that gradually increasing the number of dropped units during training leads to better accuracy and robustness to hyperparameter choices.
The effectiveness of DropBlock is further analyzed through visualizations and experiments, showing that models trained with DropBlock learn more spatially distributed representations and are more robust to changes in hyperparameters. Overall, DropBlock is a powerful tool for regularizing CNNs, especially in convolutional layers, and shows significant improvements over traditional dropout methods.