This repository contains the source code for the paper Grouped Pointwise Convolutions Significantly Reduces Parameters in EfficientNet (PDF) by Joao Paulo Schwarz Schuler, Santiago Romani, Mohamed Abdel-Nasser, Hatem Rashwan and Domenec Puig.
EfficientNet is a recent Deep Convolutional Neural Network (DCNN) architecture intended to be proportionally extendible in depth, width and resolution. Through its variants, it can achieve state of the art accuracy on the ImageNet classification task as well as on other classical challenges. Although its name refers to its efficiency with respect to the ratio between outcome (accuracy) and needed resources (number of parameters, flops), we are studying a method to reduce the original number of trainable parameters by more than 84% while keeping a very similar degree of accuracy. Our proposal is to improve the pointwise (1x1) convolutions, whose number of parameters rapidly grows due to the multiplication of the number of filters by the number of input channels that come from the previous layer. Basically, our tweak consists in grouping filters into parallel branches, where each branch processes a fraction of the input channels. However, by doing so, the learning capability of the DCNN is degraded. To avoid this effect, we suggest interleaving the output of filters from different branches at intermediate layers of consecutive pointwise convolutions. Our experiments with the CIFAR-10 dataset show that our optimized EfficientNet has similar learning capacity to the original layout when training from scratch.
You can test kEffNet on Google Colab:
All examples in this project require importing the K-CAI Neural API.
This project is an open source project. If you like what you see, please give it a star on github.
The following papers also deal about parameters and floating-point computation savings:
- Grouped Pointwise Convolutions Reduce Parameters in Convolutional Neural Networks.
- An Enhanced Scheme for Reducing the Complexity of Pointwise Convolutions in CNNs for Image Classification Based on Interleaved Grouped Filters without Divisibility Constraints.
- Reliable Deep Learning Plant Leaf Disease Classification Based on Light-Chroma Separated Branches.
- Color-aware two-branch DCNN for efficient plant disease classification.
Bibtex:
@inbook{Schuler2021k01,
author = {Schuler, Joao and Romaní, Santiago and Abdel-nasser, Mohamed and Rashwan, Hatem and Puig, Domenec},
year = {2021},
month = {10},
pages = {383-391},
title = {Grouped Pointwise Convolutions Significantly Reduces Parameters in EfficientNet},
booktitle = {Artificial Intelligence Research and Development},
publisher = {IOS Press},
isbn = {9781643682105},
doi = {10.3233/FAIA210158}
}