/Model_Compression_Paper

Apache License 2.0Apache-2.0

Model_Compression_Paper

Type of Pruning

Type F W Other
Explanation Filter pruning Weight pruning other types
Conf 2015 2016 2017 2018 2019 2020 2021
AAAI 539 548 649 938 1147 1591 1692
CVPR 602(71) 643(83) 783(71) 979(70) 1300(288) 1470(335)
NeurIPS 479 645 954 1011 1428 1900 (105)
ICLR oral-15 198 336(23) 502(24) 687 860(53)
ICML 433 621 774 1088
IJCAI 572 551 660 710 850 592
ICCV - 621 - 1077 -
ECCV 415 - 778 - 1360
MLsys

MLsys:https://proceedings.mlsys.org/paper/2019

ICCVhttps://dblp.org/db/conf/iccv/iccv2019.html

ICCV https://dblp.org/db/conf/iccv/iccv2017.html

ECCV https://link.springer.com/conference/eccv

ECCV https://zhuanlan.zhihu.com/p/157569669

CVPR https://dblp.org/db/conf/cvpr/cvpr2020.html

ICDE ECAI ACCV WACV BMVC

WACV:(Applications of Computer Vision)

量化2015 & 2016 & 2017

Title Venue Type Notion
HWGQ-Deep Learning With Low Precision by Half-wave Gaussian Quantization CVPR 孙剑
Weighted-Entropy-based Quantization for Deep Neural Networks CVPR not code
WRPN Wide Reduced-Precision Networks ICLR intel+distiller框架集成
DoReFa-Net: training low bitwidth convolutional neural networks with low bitwidth gradients ICLR 超低bit
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks ECCV 超低bit
Binaryconnect Training deep neural networks with binary weights during propagations NeurIPS 超低bit
INQ-Incremental network quantization Towards lossless cnns with low-precision weight ICLR intel

剪枝 2017

Title Venue Type Notion
Pruning Filters for Efficient ConvNets ICLR F abs(filter)
Pruning Convolutional Neural Networks for Resource Efficient Inference ICLR F 基于一阶泰勒展开近似
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression ICCV F 找一组channel近似全集
Channel pruning for accelerating very deep neural networks ICCV F LASSO回归、孙剑
Learning Efficient Convolutional Networks Through Network Slimming ICCV F 基于BN层
Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guarantee NeurIPS W 还没看
Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon NeurIPS W 还没看
Runtime Neural Pruning NeurIPS F 还没看
Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning CVPR F 还没看

量化 2018

Title Venue Type Notion
PACT: Parameterized Clipping Activation for Quantized Neural Networks ICLR
Scalable methods for 8-bit training of neural networks NeurIPS
Two-step quantization for low-bit neural networks CVPR
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference CVPR QAT和fold Bn
Joint training of low-precision neural network with quantization interval Parameters NeurIPS samsung
Lq-nets Learned quantization for highly accurate and compact deep neural networks ECCV

剪枝 2018

Title Venue Type Notion
Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers ICLR F
To prune, or not to prune: exploring the efficacy of pruning for model compression ICLR w 还没看
Discrimination-aware Channel Pruning for Deep Neural Networks NeurIPS F 还没看
Amc: Automl for model compression and acceleration on mobile devices ECCV F 还没看
Coreset-Based Neural Network Compression ECCV F 还没看
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning CVPR F 还没看
NISP: Pruning Networks using Neuron Importance Score Propagation CVPR F 还没看
Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks IJCAI F 还没看
Accelerating Convolutional Networks via Global & Dynamic Filter Pruning IJCAI F 还没看

量化 2019

Title Venue Type Notion
ACIQ-Analytical Clipping for Integer Quantization of Neural Networks ICLR
OCS-Improving Neural Network Quantization without Retraining using Outlier Channel Splitting. ICML
Data-Free Quantization Through Weight Equalization and Bias Correction ICCV(Oral)

2019

Title Venue Type Notion
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks ICLR (Best) W
Rethinking the Value of Network Pruning ICLR F
Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration CVPR (Oral) F 基于几何平均数
Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure CVPR F
Network Pruning via Transformable Architecture Search NeurIPS F
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks NeurIPS F
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask NeurIPS W
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers NeurIPS W
AutoPrune: Automatic Network Pruning by Regularizing Auxiliary Parameters NeurIPS W
Accelerate CNN via Recursive Bayesian Pruning ICCV F
Structured Pruning of Neural Networks with Budget-Aware Regularization CVPR F
Importance Estimation for Neural Network Pruning CVPR F
Dynamic Channel Pruning: Feature Boosting and Suppression ICLR F
Collaborative Channel Pruning for Deep Networks ICML F
Approximated Oracle Filter Pruning for Destructive CNN Width Optimization github ICML F

量化 2020

Title Venue Type Notion
Precision Gating Improving Neural Network Efficiency with Dynamic Dual-Precision Activations ICLR
Post-training Quantization with Multiple Points Mixed Precision without Mixed Precision ICML
Towards Unified INT8 Training for Convolutional Neural Network CVPR 商汤bp+qat
APoT-addive powers-of-two quantization an efficient non-uniform discretization for neural networks ICLR 非线性量化scheme
Post-Training Piecewise Linear Quantization for Deep Neural Networks ECCV(oral)
Training Quantized Neural Networks With a Full-Precision Auxiliary Module. CVPR(oral)

剪枝 2020

Title Venue Type Code
EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning ECCV (Oral) F PyTorch(Author)
DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation ECCV F -
DHP: Differentiable Meta Pruning via HyperNetworks ECCV F PyTorch(Author)
Towards Efficient Model Compression via Learned Global Ranking CVPR (Oral) F Pytorch(Author)
HRank: Filter Pruning using High-Rank Feature Map CVPR (Oral) F Pytorch(Author)
Differentiable Joint Pruning and Quantization for Hardware Efficiency ECCV Other -
Channel Pruning via Automatic Structure Search IJCAI F PyTorch(Author)
Proving the Lottery Ticket Hypothesis: Pruning is All You Need ICML W -
Soft Threshold Weight Reparameterization for Learnable Sparsity ICML WF Pytorch(Author)
Network Pruning by Greedy Subnetwork Selection ICML F -
Operation-Aware Soft Channel Pruning using Differentiable Masks ICML F -
DropNet: Reducing Neural Network Complexity via Iterative Pruning ICML F -
Neural Network Pruning with Residual-Connections and Limited-Data CVPR (Oral) F -
Multi-Dimensional Pruning: A Unified Framework for Model Compression CVPR (Oral) WF -
DMCP: Differentiable Markov Channel Pruning for Neural Networks CVPR (Oral) F TensorFlow(Author)
Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression CVPR F PyTorch(Author)
Few Sample Knowledge Distillation for Efficient Network Compression CVPR F -
Discrete Model Compression With Resource Constraint for Deep Neural Networks CVPR F -
Structured Compression by Weight Encryption for Unstructured Pruning and Quantization CVPR W -
Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration CVPR F -
APQ: Joint Search for Network Architecture, Pruning and Quantization Policy CVPR F -
Comparing Rewinding and Fine-tuning in Neural Network Pruning ICLR (Oral) WF TensorFlow(Author)
A Signal Propagation Perspective for Pruning Neural Networks at Initialization ICLR (Spotlight) W -
ProxSGD: Training Structured Neural Networks under Regularization and Constraints ICLR W TF+PT(Author)
One-Shot Pruning of Recurrent Neural Networks by Jacobian Spectrum Evaluation ICLR W -
Lookahead: A Far-sighted Alternative of Magnitude-based Pruning ICLR W PyTorch(Author)
Dynamic Model Pruning with Feedback ICLR WF -
Provable Filter Pruning for Efficient Neural Networks ICLR F -
Data-Independent Neural Pruning via Coresets ICLR W -
AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates AAAI F -
DARB: A Density-Aware Regular-Block Pruning for Deep Neural Networks AAAI Other -
Pruning from Scratch AAAI Other -

蒸馏 2020

Title Venue Type Notion
Neural Networks Are More Productive Teachers Than Human Raters: Active Mixup for Data-Efficient Knowledge Distillation From a Blackbox Model. CVPR(oral)

Pruning by Heyang

https://github.com/he-y/Awesome-Pruning#2018

https://github.com/MingSun-Tse/EfficientDNNs

Papers-Lottery Ticket Hypothesis (LTH)

Papers-Bayesian Compression