This paper introduces a Structured Sparsity Learning (SSL) method to regularize the structures of Deep Neural Networks (DNNs), including filters, channels, filter shapes, and layer depth. SSL aims to reduce computation cost and improve hardware efficiency by learning a compact structure from a larger DNN. The method uses group Lasso regularization during training to achieve this. Experimental results on datasets like CIFAR-10 and ImageNet show that SSL can achieve significant speedups (5.1× on CPU and 3.1× on GPU for AlexNet) while maintaining or improving classification accuracy. For example, on CIFAR-10, SSL reduces the depth of a Deep Residual Network (ResNet) from 32 layers to 18 layers without accuracy loss. The paper also discusses the effectiveness of SSL in various network structures, including LeNet, multilayer perceptrons (MLPs), ConvNet, and AlexNet, demonstrating its versatility and efficiency in both small and large-scale DNNs.This paper introduces a Structured Sparsity Learning (SSL) method to regularize the structures of Deep Neural Networks (DNNs), including filters, channels, filter shapes, and layer depth. SSL aims to reduce computation cost and improve hardware efficiency by learning a compact structure from a larger DNN. The method uses group Lasso regularization during training to achieve this. Experimental results on datasets like CIFAR-10 and ImageNet show that SSL can achieve significant speedups (5.1× on CPU and 3.1× on GPU for AlexNet) while maintaining or improving classification accuracy. For example, on CIFAR-10, SSL reduces the depth of a Deep Residual Network (ResNet) from 32 layers to 18 layers without accuracy loss. The paper also discusses the effectiveness of SSL in various network structures, including LeNet, multilayer perceptrons (MLPs), ConvNet, and AlexNet, demonstrating its versatility and efficiency in both small and large-scale DNNs.