Adaptive Mixture of Low-Rank Factorizations for Compact Neural Modeling
Implementation of "Adaptive Mixture of Low-Rank Factorizations for Compact Neural Modeling" ICLR 2019
Requires:
- pytorch >= 0.4.1
- torchvision >= 0.2.1
- tensorboard >= 1.12.0
- tensorboardX >= 1.4.0
To do list:
- Simple Network
- MLP MNIST
- MobileNet CIFAR
- MobileNet CIFAR with Low-Rank Factorization
- Different datasets support (We need to change the architecture of network)
- Implement more models with Low-Rank Factorizations
- Flops measurement
Contributors
- Denis Zuenko has implemented MobileNet_CIFAR and MobileNet_CIFAR_LowRankShit.
- Yuriy Gabuev has implemented the main idea of low-rank approximation and MLP.
- Stanislav Tsepa has implemented LowRankLayer and MLP.
- Van Khachatryan has tested experiments and made a sum up.
- Aleksandr Rubashevskii has tested experiments and made a sum up.
Reference
- "Adaptive Mixture of Low-Rank Factorizations for Compact Neural Modeling" Ting Chen, Ji Lin, Tian Lin, Song Han, Chong Wang, Dengyong Zhou, ICLR 2019