- Optimization of Convolutional Neural Networks using Quantization is an important technique to reduce the computational and memory requirements of CNN models.
- Quantization involves reducing the precision of the weights and activations of a CNN model, which results in smaller model size, reduced memory footprint, and faster inference time.
- The goal of optimization using quantization is to strike a balance between model size, accuracy, and computational efficiency.
- In this way, quantization can make it possible to deploy deep learning models on devices with limited computational resources, while maintaining acceptable performance.
The fundamental idea behind quantization is that if we convert the weights and inputs into integer types, we consume less memory and on certain hardware, the calculations are faster.
However, there is a trade-off: with quantization, we can lose significant accuracy. We will dive into this later, but first let's see why quantization works.
There are major two types of quantization techniques :
- Post-training Quantization
- Quantization-aware training
- This technique involves quantizing the weights and activations of the model from floating-point numbers to lower-precision fixed-point or integer numbers.
- This technique can be applied to any pre-trained model without retraining or fine-tuning. Its main advantage that it is simple to apply.
- Post-training quantization can reduce the model size by up to 4x and can also speed up the inference time on various hardware platforms, including CPUs, GPUs, and specialized accelerators.
- Lead to a slight degradation in model accuracy due to the loss of precision during quantization. Not suitable for models that require dynamic computations.
- Quantize the weights during training.
- First, the model is trained with full-precision weights and activations, and then fine-tuned with quantization-aware training.
- Here, even the gradients are calculated for the quantized weights.
- QAT can significantly improve the accuracy of the quantized models and reduce the accuracy loss compared to post-training quantization.
We have used the Fashion MNIST dataset.
-
Tested accuracy of different models on test data :
- Normal CNN (with convolution, pooling, dense layers)
- MobileNet
- VGG16
-
Tested accuracy on Float as well as Quantized models. Techniques used :
- Post-Training
- QAT
Model | No. of Parameters | Non-Quantized Model Accuracy | 8-bit QAT Model Accuracy | Non-Quantized Model Accuracy | 8-bit QAT Model Accuracy |
---|---|---|---|---|---|
Normal CNN | 20,426 | 88.37 % | 87.30 % | 0.9174 mb | 0.233 mb |
MobileNet | 3, 251, 259 | 85.72 % | 83.11 % | 12.71 mb | 3.36 mb |
VGG16 | 14, 781, 642 | 82.20 % | 71.82 % | 57.63 mb | 15.37 mb |
- Quantization results in a significant reduction in model size, with up to 73.33% reduction in model size observed in the case of the VGG16 model.
- In most cases, the accuracy of the 8-bit QAT (Quantization Aware Training) model is slightly lower than that of the non-quantized model. However, the drop in accuracy is relatively small and may be acceptable in resource-constrained environments.
- The effectiveness of quantization may depend on the specific architecture of the CNN model being used.