COMPRESSING DEEP CONVOLUTIONAL NETWORKS USING VECTOR QUANTIZATION

COMPRESSING DEEP CONVOLUTIONAL NETWORKS USING VECTOR QUANTIZATION

18 Dec 2014 | Yunchao Gong, Liu Liu ; Ming Yang, Lubomir Bourdev
This paper proposes a method for compressing deep convolutional neural networks (CNNs) using vector quantization to reduce storage requirements. Deep CNNs are widely used for object recognition and image classification, but their large parameter count makes them impractical for deployment on resource-limited hardware. The authors investigate vector quantization methods for compressing CNN parameters, finding that they outperform existing matrix factorization methods, especially for dense connected layers. They demonstrate that applying k-means clustering to weights or product quantization can achieve a good balance between model size and accuracy. For the ImageNet classification task, they achieve 16-24 times compression with only 1% accuracy loss using the state-of-the-art CNN. The paper focuses on compressing dense connected layers, which account for most of the storage in CNNs. They compare various vector quantization methods, including binarization, scalar quantization with k-means, product quantization, and residual quantization. They find that structured quantization methods, such as product quantization, provide additional compression gains by exploiting parameter redundancy. The results show that product quantization performs significantly better than other methods. The authors also evaluate the compressed models on image retrieval tasks, demonstrating that the compressed CNNs maintain good performance. The paper contributes to the field by systematically studying vector quantization methods for compressing CNN parameters. They show that vector quantization can achieve high compression rates without significant loss of accuracy. The results confirm the empirical findings that CNNs are over-parameterized, and that useful parameters account for about 5% of the total. The authors conclude that vector quantized CNNs can be safely applied to tasks beyond image classification. The study highlights the potential of vector quantization for compressing deep CNNs, enabling their deployment on embedded systems and mobile devices.This paper proposes a method for compressing deep convolutional neural networks (CNNs) using vector quantization to reduce storage requirements. Deep CNNs are widely used for object recognition and image classification, but their large parameter count makes them impractical for deployment on resource-limited hardware. The authors investigate vector quantization methods for compressing CNN parameters, finding that they outperform existing matrix factorization methods, especially for dense connected layers. They demonstrate that applying k-means clustering to weights or product quantization can achieve a good balance between model size and accuracy. For the ImageNet classification task, they achieve 16-24 times compression with only 1% accuracy loss using the state-of-the-art CNN. The paper focuses on compressing dense connected layers, which account for most of the storage in CNNs. They compare various vector quantization methods, including binarization, scalar quantization with k-means, product quantization, and residual quantization. They find that structured quantization methods, such as product quantization, provide additional compression gains by exploiting parameter redundancy. The results show that product quantization performs significantly better than other methods. The authors also evaluate the compressed models on image retrieval tasks, demonstrating that the compressed CNNs maintain good performance. The paper contributes to the field by systematically studying vector quantization methods for compressing CNN parameters. They show that vector quantization can achieve high compression rates without significant loss of accuracy. The results confirm the empirical findings that CNNs are over-parameterized, and that useful parameters account for about 5% of the total. The authors conclude that vector quantized CNNs can be safely applied to tasks beyond image classification. The study highlights the potential of vector quantization for compressing deep CNNs, enabling their deployment on embedded systems and mobile devices.
Reach us at info@study.space