Learning Efficient Convolutional Networks through Network Slimming

Learning Efficient Convolutional Networks through Network Slimming

22 Aug 2017 | Zhuang Liu1*, Jianguo Li2, Zhiqiang Shen3, Gao Huang4, Shoumeng Yan2, Changshui Zhang1
This paper proposes a novel learning scheme for convolutional neural networks (CNNs) called network slimming, which simultaneously reduces model size, run-time memory footprint, and computational operations without compromising accuracy. The method enforces channel-level sparsity by applying L1 regularization on the scaling factors in batch normalization (BN) layers. This allows the automatic identification and pruning of insignificant channels during training, resulting in compact models with comparable accuracy. The approach is simple to implement and requires no special software/hardware accelerators. The method is demonstrated on several state-of-the-art CNN models, including VGGNet, ResNet, and DenseNet, on various image classification datasets. For VGGNet, a multi-pass version of network slimming achieves a 20× reduction in model size and a 5× reduction in computing operations. The approach is effective in reducing computational cost while maintaining accuracy, and it can be applied to a wide range of CNN architectures. The paper also discusses related work in the field of CNN compression, including low-rank decomposition, weight quantization, weight pruning, and structured pruning. It highlights the advantages and challenges of channel-level sparsity, and introduces a simple yet effective method to achieve it by leveraging the scaling factors in BN layers. The method is shown to be effective in reducing model size, run-time memory, and computational operations, while maintaining high accuracy. Experiments on several benchmark datasets show that the proposed method can achieve up to 20× model-size compression and 5× reduction in computing operations, while maintaining the same or even higher accuracy. The method is also shown to be effective in reducing inference time and memory usage, and it can be applied to a wide range of CNN architectures. The paper concludes that network slimming is a simple and effective method for learning compact CNNs that can be deployed on resource-constrained platforms.This paper proposes a novel learning scheme for convolutional neural networks (CNNs) called network slimming, which simultaneously reduces model size, run-time memory footprint, and computational operations without compromising accuracy. The method enforces channel-level sparsity by applying L1 regularization on the scaling factors in batch normalization (BN) layers. This allows the automatic identification and pruning of insignificant channels during training, resulting in compact models with comparable accuracy. The approach is simple to implement and requires no special software/hardware accelerators. The method is demonstrated on several state-of-the-art CNN models, including VGGNet, ResNet, and DenseNet, on various image classification datasets. For VGGNet, a multi-pass version of network slimming achieves a 20× reduction in model size and a 5× reduction in computing operations. The approach is effective in reducing computational cost while maintaining accuracy, and it can be applied to a wide range of CNN architectures. The paper also discusses related work in the field of CNN compression, including low-rank decomposition, weight quantization, weight pruning, and structured pruning. It highlights the advantages and challenges of channel-level sparsity, and introduces a simple yet effective method to achieve it by leveraging the scaling factors in BN layers. The method is shown to be effective in reducing model size, run-time memory, and computational operations, while maintaining high accuracy. Experiments on several benchmark datasets show that the proposed method can achieve up to 20× model-size compression and 5× reduction in computing operations, while maintaining the same or even higher accuracy. The method is also shown to be effective in reducing inference time and memory usage, and it can be applied to a wide range of CNN architectures. The paper concludes that network slimming is a simple and effective method for learning compact CNNs that can be deployed on resource-constrained platforms.
Reach us at info@study.space
Understanding Learning Efficient Convolutional Networks through Network Slimming