/ReadingPaper_Daily

Model_Compression_Paper

Type of Pruning

Type F W Other
Explanation Filter pruning Weight pruning other types
Conf 2015 2016 2017 2018 2019 2020 2021
AAAI 539 548 649 938 1147 1591 1692
ICLR oral-15 198 336(23) 502(24) 687 860(53)
CVPR 602(71) 643(83) 783(71) 979(70) 1300(288) 1470(335) Feb.28(7015)
NeurIPS 479 645 954 1011 1428 1900 (105)
ICML 433 621 774 1088 May.8th
IJCAI 572 551 660 710 850 592
ICCV - 621 - 1077 -
ECCV 415 - 778 - 1360
MLsys
ISCA 57 54 54 63 62 77
ECAI - 562 - 656 - 365

MLsys:https://proceedings.mlsys.org/paper/2019

ICCVhttps://dblp.org/db/conf/iccv/iccv2019.html

ICCV https://dblp.org/db/conf/iccv/iccv2017.html

ECCV https://link.springer.com/conference/eccv

ECCV https://zhuanlan.zhihu.com/p/157569669

CVPR https://dblp.org/db/conf/cvpr/cvpr2020.html

ICDE ACCV WACV BMVC

WACV:(Applications of Computer Vision)

nsdi sigcomm osdi sosp sigmod mobicom sosp ATC MLsys

2021:

  1. Diversifying Sample Generation for Accurate Data-Free Quantization
  2. BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction
  3. Learnable Companding Quantization for Accurate Low-bit Neural Networks
  4. UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems
  5. Distribution Adaptive INT8 Quantization for Training CNNs
  6. Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillation
  7. You Only Look One-level Feature
  8. Probabilistic two-stage detection
  9. General Instance Distillation for Object Detection
  10. Generalized Focal Loss V2: Learning Reliable Localization Quality Estimation for Dense Object Detection
  11. Sparse R-CNN: End-to-End Object Detection with Learnable Proposals
  12. Relation Networks for Object Detection
  13. RelationNet++: Bridging Visual Representations for Object Detection via Transformer Decoder
  14. Weighted boxes fusion: Ensembling boxes from different object detection models
  15. Dert：End-to-End Object Detection with Transformers
  16. Fine-grained Angular Contrastive Learning with Coarse Labels(😮oral) 使用自监督进行 Coarse Labels（粗标签）的细粒度分类方面的工作。粗标签与细粒度标签相比，更容易和更便宜，因为细粒度标签通常需要域专家。
  17. UP-DETR: Unsupervised Pre-training for Object Detection with Transformers(oral)
  18. ReXNet: Diminishing Representational Bottleneck on Convolutional Neural Network
  19. RepVGG: Making VGG-style ConvNets Great Again
  20. Revisiting Dynamic Convolution via Matrix Decomposition
  21. An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detection
  22. CenterMask : Real-Time Anchor-Free Instance Segmentation
  23. VarGNet: Variable Group Convolutional Neural Network for Efficient Embedded Computing
  24. Manifold Regularized Dynamic Network Pruning
  25. Fast and Accurate Model Scaling
  26. Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
  27. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
  28. On Implicit Filter Level Sparsity in Convolutional Neural Networks 29.Skip-Convolutions for Efficient Video Processing CVPR2021 30.FIXUP INITIALIZATION: RESIDUAL LEARNING WITHOUT NORMALIZATION ICLR 2019 31.ReZero is All You Need: Fast Convergence at Large Depth 2020.May 32.Going deeper with Image Transformers 2021
  29. Batch Normalization Biases Residual Blocks Towards the Identity Function in Deep Networks NeurIPS 2020 34.Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting AAAI2021 35.WaveNet
稀疏化

2019 SeerNet: Predicting Convolutional Neural Network Feature-Map Sparsity Through Low-Bit Quantization CVPR

量化2021-(3)
  1. Any-Precision Deep Neural Networks AAAI
  2. Post-training Quantization with Multiple Points Mixed Precision without Mixed Precision | AAAI Mixed Precision
  3. CPT:Efficient deep neural network training via cyclic precision ICLR

量化 2020-(14)

  1. Precision Gating Improving Neural Network Efficiency with Dynamic Dual-Precision Activations ICLR
  2. Post-training Quantization with Multiple Points Mixed Precision without Mixed Precision ICML
  3. Towards Unified INT8 Training for Convolutional Neural Network | CVPR 商汤bp+qat
  4. APoT-addive powers-of-two quantization an efficient non-uniform discretization for neural networks ICLR 非线性量化scheme
  5. Post-Training Piecewise Linear Quantization for Deep Neural Networks ECCV(oral)
  6. Training Quantized Neural Networks With a Full-Precision Auxiliary Module CVPR(oral)
  7. MCUNet: Tiny Deep Learning on IoT Devices NeurIPS
  8. HAWQ-V2 Hessian Aware trace-Weighted Quantization of Neural Networks NeurIPS
  9. HAWQ-V3: Dyadic Neural Network Quantization
  10. Subtensor Quantization for Mobilenets - Mobilenets
  11. Generative Low-bitwidth Data Free Quantization ECCV GAN
剪枝 2020
  1. EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning ECCV(Oral) F PyTorch(Author) DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation ECCV F
  2. AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates AAAI F -
  3. Pruning from Scratch AAAI Other -
  4. DHP: Differentiable Meta Pruning via HyperNetworks ECCV F PyTorch(Author)
  5. Towards Efficient Model Compression via Learned Global Ranking CVPR(Oral) F Pytorch(Author)
  6. HRank: Filter Pruning using High-Rank Feature Map CVPR(Oral) F 可
  7. Soft Threshold Weight Reparameterization for Learnable Sparsity ICML WF Pytorch(Author)
  8. Network Pruning by Greedy Subnetwork Selection ICML F -
  9. Operation-Aware Soft Channel Pruning using Differentiable Masks ICML F Mask
量化 2019-(22)

  1. ACIQ-Analytical Clipping for Integer Quantization of Neural Networks ICLR

  2. Differentiable Quantization of Deep Neural Networks NeurIPS 没代码+NAS

  3. Post training 4-bit quantization of convolutional networks for rapid-deployment NeurIPS ACIQ

  4. Data-Free Quantization Through Weight Equalization and Bias Correction ICCV(Oral)

  5. Data-Free Quantization Through Weight Equalization and Bias Correction ICCV

  6. HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision ICCV(Poster) 可微分

  7. **(DSQ)**Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks ICCV 可微分

  8. Low-bit Quantization of Neural Networks for Efficient Inference ICCV Workshops 没代码

  9. Quantization Networks CVPR 可微分

  10. Fully Quantized Network for Object Detection CVPR 没代码

  11. HAQ Hardware-Aware Automated Quantization With Mixed Precision CVPR RL

  12. Accelerating Convolutional Neural Networks via Activation Map Compression CVPR 没代码

  13. Learning to quantize deep networks by optimizing quantization intervals with task loss CVPR 可微分

  14. Accelerating Convolutional Neural Networks via Activation Map Compression CVPR 没看懂pipeline

  15. Fighting Quantization Bias With Bias CVPR W 给量化误差补偿bias

  16. Learning low-precision neural networks without Straight-Through Estimator(STE) IJCAI 可微分

  17. OCS-Improving Neural Network Quantization without Retraining using Outlier Channel Splitting. | ICML

  18. Same, Same But Different Recovering Neural Network Quantization Error Through Weight Factorization ICML 与高通的DFQ很像

  19. Learning low-precision neural networks without Straight-Through Estimator (STE) IJCAI 没代码+可微分

  20. SeerNet Predicting Convolutional Neural Network Feature-Map Sparsity Through Low-Bit Quantization ECCV 稀疏化

  21. DAC Data-free Automatic Acceleration of Convolutional Networks WACV DW Conv

  22. A Quantization-Friendly Separable Convolution for MobileNets - MobileNets

剪枝 2019

  1. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks ICLR(Best) W winning ticket

  2. Rethinking the Value of Network Pruning ICLR F slim prune

  3. Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration CVPR (Oral) F 基于几何平均数

  4. Importance Estimation for Neural Network Pruning CVPR F Nvidia

  5. Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure CVPR F 聚类

量化 2018-(11)
  1. PACT: Parameterized Clipping Activation for Quantized Neural Networks | ICLR
  2. Scalable methods for 8-bit training of neural networks | NeurIPS
  3. Two-step quantization for low-bit neural networks | CVPR
  4. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference | CVPR **QAT和fold Bn**
  5. Joint training of low-precision neural network with quantization interval Parameters | NeurIPS
  6. Lq-nets Learned quantization for highly accurate and compact deep neural networks | ECCV
  7. Apprentice Using KD Techniques to Improve Low-Precision Network Accuracy | ICLR
  8. calable Methods for 8-bit Training of Neura Network | NeurIPS | |
  9. Quantization mimic Towards very tiny cnn for object detection | ECCV | | KD+量化
  10. Mimicking very efficient network for object detection | CVPR | | 跟上面
  11. Training and inference with integers in deep neural networks | ICLR | | WAGE
剪枝 2018
  1. Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers | ICLR | F | ISAT+质疑了norm-based
  2. A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers | ECCV | w | ADMM
  3. Amc: Automl for model compression and acceleration on mobile devices | ECCV | F | 还没看
  4. Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks | IJCAI | F | 剪枝后还可以恢复
  5. Data-Driven Sparse Structure Selection for Deep Neural Networks | ECCV | F | APG +Bn
剪枝 2017
  1. Pruning Filters for Efficient ConvNets | ICLR | F | abs(filter)
  2. Pruning Convolutional Neural Networks for Resource Efficient Inference | ICLR | F | 基于一阶泰勒展开近似
  3. ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression | ICCV | F | 找一组channel近似全集
  4. Channel pruning for accelerating very deep neural networks | ICCV | F | LASSO回归、孙剑
  5. Learning Efficient Convolutional Networks Through Network Slimming | ICCV | F | 基于BN层
  6. Runtime Neural Pruning | NeurIPS | | Markov+RL
  7. Network trimming A data-driven neuron pruning approach towards efficient deep architectures | NeurIPS | | APoZ
量化2015 & 2016 & 2017-(8)
  1. HWGQ-Deep Learning With Low Precision by Half-wave Gaussian Quantization | CVPR | | 孙剑
  2. Weighted-Entropy-based Quantization for Deep Neural Networks | CVPR | | not code |
  3. WRPN Wide Reduced-Precision Networks | ICLR | | intel+distiller框架集成 |
  4. DoReFa-Net: training low bitwidth convolutional neural networks with low bitwidth gradients | ICLR | | 超低bit
  5. XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks | ECCV | | 超低bit
  6. Binaryconnect Training deep neural networks with binary weights during propagations | NeurIPS | | 超低bit
  7. INQ-Incremental network quantization Towards lossless cnns with low-precision weight | ICLR | | intel
  8. Convolutional Neural Networks using Logarithmic Data Representation | ICML | | scheme