Awesome Model Quantization

This repo collects papers, docs, codes about model quantization for anyone who wants to do research on it. We are continuously improving the project. Welcome to PR the works (papers, repositories) that are missed by the repo.

Benchmark
Survey Papers
- Survey of Binarization
- Survey of Quantization
Papers
- 2021
- 2020
- 2019
- 2018
- 2017
- Ealier
Codes and Docs
Our Team

Benchmark

MQBench

The paper MQBench: Towards Reproducible and Deployable Model Quantization Benchmark (NeurIPS 2021) is a benchmark and framework for evluating the quantization algorithms under real world hardware deployments. For details, please refer to:

MQBench: Towards Reproducible and Deployable Model Quantization Benchmark [Paper] [Project]

Yuhang Li, Mingzhu Shen, Jian Ma, Yan Ren, Mingxin Zhao, Qi Zhang, Ruihao Gong, Fengwei Yu, Junjie Yan.

Bibtex

@article{2021MQBench,
  title = "MQBench: Towards Reproducible and Deployable Model Quantization Benchmark",
  author= "Yuhang Li* and Mingzhu Shen* and Jian Ma* and Yan Ren* and Mingxin Zhao* and Qi Zhang* and Ruihao Gong and Fengwei Yu and Junjie Yan",
  journal = "https://openreview.net/forum?id=TUplOmF8DsM",
  year = "2021"
}

Survey_Papers

Survey_of_Binarization

Our survey paper Binary Neural Networks: A Survey (Pattern Recognition) is a comprehensive survey of recent progress in binary neural networks. For details, please refer to:

Binary Neural Networks: A Survey [Paper] [Blog]

Haotong Qin, Ruihao Gong, Xianglong Liu*, Xiao Bai, Jingkuan Song, and Nicu Sebe.

Bibtex

@article{Qin:pr20_bnn_survey,
	title = "Binary neural networks: A survey",
	author = "Haotong Qin and Ruihao Gong and Xianglong Liu and Xiao Bai and Jingkuan Song and Nicu Sebe",
	journal = "Pattern Recognition",
	volume = "105",
	pages = "107281",
	year = "2020"
}

Survey_of_Quantization

The survey paper A Survey of Quantization Methods for Efficient Neural Network Inference (ArXiv) is a comprehensive survey of recent progress in quantization. For details, please refer to:

A Survey of Quantization Methods for Efficient Neural Network Inference [Paper]

Amir Gholami* , Sehoon Kim* , Zhen Dong* , Zhewei Yao* , Michael W. Mahoney, Kurt Keutzer. (* Equal contribution)

Bibtex

@misc{gholami2021survey,
      title={A Survey of Quantization Methods for Efficient Neural Network Inference}, 
      author={Amir Gholami and Sehoon Kim and Zhen Dong and Zhewei Yao and Michael W. Mahoney and Kurt Keutzer},
      year={2021},
      eprint={2103.13630},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Papers

Statistics: 🔥 highly cited | ⭐ code is available and star > 50

2021

[ACM MM] VQMG: Hierarchical Vector Quantised and Multi-hops Graph Reasoning for Explicit Representation Learning.[other]
[ACM MM] Fully Quantized Image Super-Resolution Networks. [low-bit]
[NeurIPS] Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples. [low-bit]
[NeurIPS] Post-Training Quantization for Vision Transformer. [mixed]
[NeurIPS] Post-Training Sparsity-Aware Quantization. [low-bit]
[NeurIPS] Divergence Frontiers for Generative Models: Sample Complexity, Quantization Effects, and Frontier Integrals.
[NeurIPS] VQ-GNN: A Universal Framework to Scale up Graph Neural Networks using Vector Quantization. [other]
[NeurIPS] Qu-ANTI-zation: Exploiting Quantization Artifacts for Achieving Adversarial Outcomes .
[CVPR] Diversifying Sample Generation for Accurate Data-Free Quantization. [low-bit]
[CVPR] Learnable Companding Quantization for Accurate Low-bit Neural Networks. [low-bit]
[CVPR] Zero-shot Adversarial Quantization. [low-bit] [torch]
[CVPR] Binary Graph Neural Networks. [binarization] [torch]
[CVPR] Network Quantization with Element-wise Gradient Scaling. [low-bit] [torch]
[ICLR] BiPointNet: Binary Neural Network for Point Clouds. [binarization] [torch]
[ICLR] Reducing the Computational Cost of Deep Generative Models with Binary Neural Networks. [binarization]
[ICLR] High-Capacity Expert Binary Networks. [binarization]
[ICLR] Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network. [binarization]
[ICLR] BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction. [low-bit] [torch]
[ICLR] Neural gradients are near-lognormal: improved quantized and sparse training. [low-bit]
[ICLR] Training with Quantization Noise for Extreme Model Compression. [low-bit]
[ICLR] Incremental few-shot learning via vector quantization in deep embedded space. [low-bit]
[ICLR] Degree-Quant: Quantization-Aware Training for Graph Neural Networks. [low-bit]
[ICLR] BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantization. [low-bit]
[ICLR] Simple Augmentation Goes a Long Way: ADRL for DNN Quantization. [low-bit]
[ICLR] Sparse Quantized Spectral Clustering. [low-bit]
[ICLR] WrapNet: Neural Net Inference with Ultra-Low-Resolution Arithmetic. [low-bit]
[ECCV] PAMS: Quantized Super-Resolution via Parameterized Max Scale. [low-bit]
[AAAI] Distribution Adaptive INT8 Quantization for Training CNNs. [low-bit]
[AAAI] Stochastic Precision Ensemble: Self‐Knowledge Distillation for Quantized Deep Neural Networks. [low-bit]
[AAAI] Optimizing Information Theory Based Bitwise Bottlenecks for Efficient Mixed-Precision Activation Quantization. [low-bit]
[AAAI] OPQ: Compressing Deep Neural Networks with One-shot Pruning-Quantization. [low-bit]
[AAAI] Scalable Verification of Quantized Neural Networks. [low-bit]
[AAAI] Uncertainty Quantification in CNN through the Bootstrap of Convex Neural Networks. [low-bit]
[AAAI] FracBits: Mixed Precision Quantization via Fractional Bit-Widths. [low-bit]
[AAAI] Post-‐training Quantization with Multiple Points: Mixed Precision without Mixed Precision. [low-bit]
[AAAI] Vector Quantized Bayesian Neural Network Inference for Data Streams. [low-bit]
[AAAI] TRQ: Ternary Neural Networks with Residual Quantization. [low-bit]
[AAAI] Memory and Computation-Efficient Kernel SVM via Binary Embedding and Ternary Coefficients. [binarization]
[AAAI] Compressing Deep Convolutional Neural Networks by Stacking Low-Dimensional Binary Convolution Filters. [binarization]
[AAAI] Training Binary Neural Network without Batch Normalization for Image Super-Resolution. [binarization]
[AAAI] SA-BNN: State-Aware Binary Neural Network. [binarization]
[ACL] On the Distribution, Sparsity, and Inference-time Quantization of Attention Values in Transformers. [low-bit]
[arxiv] Any-Precision Deep Neural Networks. [mixed] [torch]
[arxiv] ReCU: Reviving the Dead Weights in Binary Neural Networks. [binarization] [torch]
[arxiv] Post-Training Quantization for Vision Transformer. [nlp]
[arxiv] A Survey of Quantization Methods for Efficient Neural Network Inference.

2020

[ACL] End to End Binarized Neural Networks for Text Classification. [binarization]
[AAAI] HLHLp: Quantized Neural Networks Traing for Reaching Flat Minima in Loss Sufrface. [low-bit]
[AAAI] [72:fire:] Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT. [low-bit] [nlp]
[AAAI] Sparsity-Inducing Binarized Neural Networks. [binarization]
[AAAI] Towards Accurate Low Bit-Width Quantization with Multiple Phase Adaptations.
[COOL CHIPS] A Novel In-DRAM Accelerator Architecture for Binary Neural Network. [hardware]
[CoRR] Training Binary Neural Networks using the Bayesian Learning Rule. [binarization]
[CVPR] [47:fire:] GhostNet: More Features from Cheap Operations. [low-bit] [tensorflow & torch] [1.2k:star:]
[CVPR] Forward and Backward Information Retention for Accurate Binary Neural Networks. [binarization] [torch] [105:star:]
[CVPR] APQ: Joint Search for Network Architecture, Pruning and Quantization Policy. [low-bit] [torch] [76:star:]
[CVPR] Rotation Consistent Margin Loss for Efficient Low-Bit Face Recognition. [low-bit]
[CVPR] BiDet: An Efficient Binarized Object Detector. [ binarization ] [torch] [112:star:]
[CVPR] Fixed-Point Back-Propagation Training. [video] [low-bit]
[CVPR] Low-Bit Quantization Needs Good Distribution. [low-bit]
[DATE] BNNsplit: Binarized Neural Networks for embedded distributed FPGA-based computing systems. [binarization]
[DATE] PhoneBit: Efficient GPU-Accelerated Binary Neural Network Inference Engine for Mobile Phones. [binarization] [hardware]
[DATE] OrthrusPE: Runtime Reconfigurable Processing Elements for Binary Neural Networks. [binarization]
[ECCV] Learning Architectures for Binary Networks. [binarization] [torch]
[ECCV]PROFIT: A Novel Training Method for sub-4-bit MobileNet Models. [low-bit]
[ECCV] ProxyBNN: Learning Binarized Neural Networks via Proxy Matrices. [binarization]
[ECCV] ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions. [binarization] [torch] [108:star:]
[ECCV] Differentiable Joint Pruning and Quantization for Hardware Efficiency. [hardware]
[EMNLP] TernaryBERT: Distillation-aware Ultra-low Bit BERT. [low-bit] [nlp]
[EMNLP] Fully Quantized Transformer for Machine Translation. [low-bit] [nlp]
[ICET] An Energy-Efficient Bagged Binary Neural Network Accelerator. [binarization] [hardware]
[ICASSP] Balanced Binary Neural Networks with Gated Residual. [binarization]
[ICML] Training Binary Neural Networks through Learning with Noisy Supervision. [binarization]
[ICLR] DMS: Differentiable Dimension Search for Binary Neural Networks. [binarization]
[ICLR] [19:fire:] Training Binary Neural Networks with Real-to-Binary Convolutions. [binarization] [code is comming] [re-implement]
[ICLR] BinaryDuo: Reducing Gradient Mismatch in Binary Activation Network by Coupling Binary Activations. [binarization] [torch]
[ICLR] Mixed Precision DNNs: All You Need is a Good Parametrization. [mixed] [code] [73:star:]
[IJCV] Binarized Neural Architecture Search for Efficient Object Recognition. [binarization]
[IJCAI] CP-NAS: Child-Parent Neural Architecture Search for Binary Neural Networks. [binarization]
[IJCAI] Towards Fully 8-bit Integer Inference for the Transformer Model. [low-bit] [nlp]
[IJCAI] Soft Threshold Ternary Networks. [low-bit]
[IJCAI] Overflow Aware Quantization: Accelerating Neural Network Inference by Low-bit Multiply-Accumulate Operations. [low-bit]
[IJCAI] Direct Quantization for Training Highly Accurate Low Bit-width Deep Neural Networks. [low-bit]
[IJCAI] Fully Nested Neural Network for Adaptive Compression and Quantization. [low-bit]
[ISCAS] MuBiNN: Multi-Level Binarized Recurrent Neural Network for EEG Signal Classification. [binarization]
[ISQED] BNN Pruning: Pruning Binary Neural Network Guided by Weight Flipping Frequency. [binarization] [torch]
[MICRO] GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference. [low-bit] [nlp]
[MLST] Compressing deep neural networks on FPGAs to binary and ternary precision with HLS4ML. [hardware] [binarization] [low-bit]
[NeurIPS] Rotated Binary Neural Network. [binarization] [torch]
[NeurIPS] Searching for Low-Bit Weights in Quantized Neural Networks. [low-bit] [torch]
[NeurIPS] Universally Quantized Neural Compression. [low-bit]
[NeurIPS] Efficient Exact Verification of Binarized Neural Networks. [binarization] [torch]
[NeurIPS] Path Sample-Analytic Gradient Estimators for Stochastic Binary Networks. [binarization] [code]
[NeurIPS] HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks. [low-bit]
[NeurIPS] Bayesian Bits: Unifying Quantization and Pruning. [low-bit]
[NeurIPS] Robust Quantization: One Model to Rule Them All. [low-bit]
[NeurIPS] Closing the Dequantization Gap: PixelCNN as a Single-Layer Flow. [low-bit] [torch]
[NeurIPS] Adaptive Gradient Quantization for Data-Parallel SGD. [low-bit] [torch]
[NeurIPS] FleXOR: Trainable Fractional Quantization. [low-bit]
[NeurIPS] Position-based Scaled Gradient for Model Quantization and Pruning. [low-bit] [torch]
[NN] Training high-performance and large-scale deep neural networks with full 8-bit integers. [low-bit]
[Neurocomputing] Eye localization based on weight binarization cascade convolution neural network. [binarization]
[PR] [23:fire:] Binary neural networks: A survey. [binarization]
[PR Letters] Controlling information capacity of binary neural network. [binarization]
[SysML] Riptide: Fast End-to-End Binarized Neural Networks. [low-bit] [tensorflow] [129:star:]
[TPAMI] Hierarchical Binary CNNs for Landmark Localization with Limited Resources. [binarization] [homepage] [code]
[TPAMI] Deep Neural Network Compression by In-Parallel Pruning-Quantization.
[TPAMI] Towards Efficient U-Nets: A Coupled and Quantized Approach.
[TVLSI] Phoenix: A Low-Precision Floating-Point Quantization Oriented Architecture for Convolutional Neural Networks. [low-bit]
[WACV] MoBiNet: A Mobile Binary Network for Image Classification. [binarization]
[IEEE Access] An Energy-Efficient and High Throughput in-Memory Computing Bit-Cell With Excellent Robustness Under Process Variations for Binary Neural Network. [binarization] [hardware]
[IEEE Trans. Magn] SIMBA: A Skyrmionic In-Memory Binary Neural Network Accelerator. [binarization]
[IEEE TCS.II] A Resource-Efficient Inference Accelerator for Binary Convolutional Neural Networks. [hardware]
[IEEE TCS.I] IMAC: In-Memory Multi-Bit Multiplication and ACcumulation in 6T SRAM Array. [low-bit]
[IEEE Trans. Electron Devices] Design of High Robustness BNN Inference Accelerator Based on Binary Memristors. [binarization] [hardware]
[arxiv] Training with Quantization Noise for Extreme Model Compression. [low-bit] [torch]
[arxiv] Binarized Graph Neural Network. [binarization]
[arxiv] How Does Batch Normalization Help Binary Training? [binarization]
[arxiv] Distillation Guided Residual Learning for Binary Convolutional Neural Networks. [binarization]
[arxiv] Accelerating Binarized Neural Networks via Bit-Tensor-Cores in Turing GPUs. [binarization] [code]
[arxiv] MeliusNet: Can Binary Neural Networks Achieve MobileNet-level Accuracy? [binarization] [code] [192:star:]
[arxiv] RPR: Random Partition Relaxation for Training; Binary and Ternary Weight Neural Networks. [binarization] [low-bit]
[paper] Towards Lossless Binary Convolutional Neural Networks Using Piecewise Approximation. [binarization]
[arxiv] Understanding Learning Dynamics of Binary Neural Networks via Information Bottleneck. [binarization]
[arxiv] BinaryBERT: Pushing the Limit of BERT Quantization. [binarization] [nlp]
[ECCV] BATS: Binary ArchitecTure Search. [binarization]

2019

[AAAI] Efficient Quantization for Neural Networks with Binary Weights and Low Bitwidth Activations. [low-bit] [binarization]
[AAAI] [31:fire:] Projection Convolutional Neural Networks for 1-bit CNNs via Discrete Back Propagation. [binarization]
[APCCAS] Using Neuroevolved Binary Neural Networks to solve reinforcement learning environments. [binarization] [code]
[BMVC] [32:fire:] XNOR-Net++: Improved Binary Neural Networks. [binarization]
[BMVC] Accurate and Compact Convolutional Neural Networks with Trained Binarization. [binarization]
[CoRR] RBCN: Rectified Binary Convolutional Networks for Enhancing the Performance of 1-bit DCNNs. [binarization]
[CoRR] TentacleNet: A Pseudo-Ensemble Template for Accurate Binary Convolutional Neural Networks. [binarization]
[CoRR] Improved training of binary networks for human pose estimation and image recognition. [binarization]
[CoRR] Binarized Neural Architecture Search. [binarization]
[CoRR] Matrix and tensor decompositions for training binary neural networks. [binarization]
[CoRR] Back to Simplicity: How to Train Accurate BNNs from Scratch? [binarization] [code] [193:star:]
[CVPR] [53:fire:] Structured Binary Neural Networks for Accurate Image Classification and Semantic Segmentation. [binarization]
[CVPR] SeerNet: Predicting Convolutional Neural Network Feature-Map Sparsity Through Low-Bit Quantization. [low-bit]
[CVPR] [218:fire:] HAQ: Hardware-Aware Automated Quantization with Mixed Precision. [low-bit] [hardware] [torch] [233:star:]
[CVPR] [48:fire:] Quantization Networks. [binarization] [torch] [82:star:]
[CVPR] Fully Quantized Network for Object Detection. [low-bit]
[CVPR] Learning Channel-Wise Interactions for Binary Convolutional Neural Networks. [binarization]
[CVPR] [31:fire:] Circulant Binary Convolutional Networks: Enhancing the Performance of 1-bit DCNNs with Circulant Back Propagation. [binarization]
[CVPR] [36:fire:] Regularizing Activation Distribution for Training Binarized Deep Networks. [binarization]
[CVPR] A Main/Subsidiary Network Framework for Simplifying Binary Neural Network. [binarization]
[CVPR] Binary Ensemble Neural Network: More Bits per Network or More Networks per Bit? [binarization]
[FPGA] Towards Fast and Energy-Efficient Binarized Neural Network Inference on FPGA. [binarization] [hardware]
[GLSVLSI] Binarized Depthwise Separable Neural Network for Object Tracking in FPGA. [binarization] [hardware]
[ICCV] [55:fire:] Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks. [low-bit]
[ICCV] Bayesian optimized 1-bit cnns. [binarization]
[ICCV] Searching for Accurate Binary Neural Architectures. [binarization]
[ICCV] Data-Free Quantization Through Weight Equalization and Bias Correction. [low-bit] [hardware] [torch]
[ICML] Efficient 8-Bit Quantization of Transformer Neural Machine Language Translation Model. [low-bit] [nlp]
[ICLR] [37:fire:] ProxQuant: Quantized Neural Networks via Proximal Operators. [binarization] [low-bit] [torch]
[ICLR] An Empirical study of Binary Neural Networks' Optimisation. [binarization]
[ICIP] Training Accurate Binary Neural Networks from Scratch. [binarization] [code] [192:star:]
[ICUS] Balanced Circulant Binary Convolutional Networks. [binarization]
[IJCAI] Binarized Neural Networks for Resource-Efficient Hashing with Minimizing Quantization Loss. [binarization]
[IJCAI] Binarized Collaborative Filtering with Distilling Graph Convolutional Network. [binarization]
[IJCAI] Binarized Collaborative Filtering with Distilling Graph Convolutional Networks. [binarization]
[ISOCC] Dual Path Binary Neural Network. [binarization]
[IEEE J. Emerg. Sel. Topics Circuits Syst.] Hyperdrive: A Multi-Chip Systolically Scalable Binary-Weight CNN Inference Engine. [hardware]
[IEEE JETC] [128:fire:] Eyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices. [hardware]
[IEEE J. Solid-State Circuits] An Energy-Efficient Reconfigurable Processor for Binary-and Ternary-Weight Neural Networks With Flexible Data Bit Width. [binarization] [low-bit]
[MDPI Electronics] A Review of Binarized Neural Networks. [binarization]
[NeurIPS] MetaQuant: Learning to Quantize by Learning to Penetrate Non-differentiable Quantization. [low-bit] [torch]
[NeurIPS] Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization. [binarization] [tensorflow]
[NeurIPS] [43:fire:] Regularized Binary Network Training. [binarization]
[NeurIPS] [44:fire:] Q8BERT: Quantized 8Bit BERT. [low-bit] [nlp]
[NeurIPS] Fully Quantized Transformer for Improved Translation. [low-bit] [nlp]
[NeurIPS] Normalization Helps Training of Quantized LSTM. [low-bit] [binarization]
[RoEduNet] PXNOR: Perturbative Binary Neural Network. [binarization] [code]
[SiPS] Knowledge distillation for optimization of quantized deep neural networks. [low-bit]
[TMM] [45:fire:] Deep Binary Reconstruction for Cross-Modal Hashing. [binarization]
[TMM] Compact Hash Code Learning With Binary Deep Neural Network. [binarization]
[TMM] Compact Hash Code Learning With Binary Deep Neural Network. [binarization]
[IEEE TCS.I] Xcel-RAM: Accelerating Binary Neural Networks in High-Throughput SRAM Compute Arrays. [hardware]
[IEEE TCS.I] Recursive Binary Neural Network Training Model for Efficient Usage of On-Chip Memory. [binarization]
[VLSI-SoC] A Product Engine for Energy-Efficient Execution of Binary Neural Networks Using Resistive Memories. [binarization] [hardware]
[paper] [43:fire:] BNN+: Improved Binary Network Training. [binarization]
[arxiv] Self-Binarizing Networks. [binarization]
[arxiv] Towards Unified INT8 Training for Convolutional Neural Network. [low-bit]
[arxiv] daBNN: A Super Fast Inference Framework for Binary Neural Networks on ARM devices. [binarization] [hardware] [code]
[arxiv] QKD: Quantization-aware Knowledge Distillation. [low-bit]
[arxiv] [59:fire:] Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search. [low-bit]

2018

[AAAI] From Hashing to CNNs: Training BinaryWeight Networks via Hashing. [binarization]
[AAAI] [136:fire:] Extremely Low Bit Neural Network: Squeeze the Last Bit Out with ADMM. [low-bit] [homepage]
[CAAI] Fast object detection based on binary deep convolution neural networks. [binarization]
[CoRR] LightNN: Filling the Gap between Conventional Deep Neural Networks and Binarized Networks. [binarization]
[CoRR] BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weights. [binarization]
[CVPR] [63:fire:] Two-Step Quantization for Low-bit Neural Networks. [low-bit]
[CVPR] Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activations. [low-bit]
[CVPR] [97:fire:] Towards Effective Low-bitwidth Convolutional Neural Networks. [low-bit]
[CVPR] Modulated convolutional networks. [binarization]
[CVPR] [67:fire:] SYQ: Learning Symmetric Quantization For Efficient Deep Neural Networks. [low-bit] [code]
[CVPR] [630:fire:] Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. [low-bit]
[ECCV] Training Binary Weight Networks via Semi-Binary Decomposition. [binarization]
[ECCV] [47:fire:] TBN: Convolutional Neural Network with Ternary Inputs and Binary Weights. [binarization] [low-bit] [torch]
[ECCV] [202:fire:] LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks. [low-bit] [tensorflow] [188:star:]
[ECCV] [145:fire:] Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm. [binarization] [torch] [120:star:]
[FCCM] ReBNet: Residual Binarized Neural Network. [binarization] [tensorflow]
[FPL] FBNA: A Fully Binarized Neural Network Accelerator. [hardware]
[ICLR] [65:fire:] Loss-aware Weight Quantization of Deep Networks. [low-bit] [code]
[ICLR] [230:fire:] Model compression via distillation and quantization. [low-bit] [torch] [284:star:]
[ICLR] [201:fire:] PACT: Parameterized Clipping Activation for Quantized Neural Networks. [low-bit]
[ICLR] [168:fire:] WRPN: Wide Reduced-Precision Networks. [low-bit]
[ICLR] Analysis of Quantized Models. [low-bit]
[ICLR] [141:fire:] Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy. [low-bit]
[IJCAI] Deterministic Binary Filters for Convolutional Neural Networks. [binarization]
[IJCAI] Planning in Factored State and Action Spaces with Learned Binarized Neural Network Transition Models. [binarization]
[IJCNN] Analysis and Implementation of Simple Dynamic Binary Neural Networks. [binarization]
[IPDPS] BitFlow: Exploiting Vector Parallelism for Binary Neural Networks on CPU. [binarization]
[IEEE J. Solid-State Circuits] [66:fire:] BRein Memory: A Single-Chip Binary/Ternary Reconfigurable in-Memory Deep Neural Network Accelerator Achieving 1.4 TOPS at 0.6 W. [hardware] [low-bit] [binarization]
[NCA] [88:fire:] A survey of FPGA-based accelerators for convolutional neural networks. [hardware]
[NeurIPS] [150:fire:] Training Deep Neural Networks with 8-bit Floating Point Numbers. [low-bit]
[NeurIPS] [91:fire:] Scalable methods for 8-bit training of neural networks. [low-bit] [torch]
[MM] BitStream: Efficient Computing Architecture for Real-Time Low-Power Inference of Binary Neural Networks on CPUs. [binarization]
[Res Math Sci] Blended coarse gradient descent for full quantization of deep neural networks. [low-bit] [binarization]
[TCAD] XNOR Neural Engine: A Hardware Accelerator IP for 21.6-fJ/op Binary Neural Network Inference. [hardware]
[TRETS] [50:fire:] FINN-R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks. [low-bit]
[TVLSI] An Energy-Efficient Architecture for Binary Weight Convolutional Neural Networks. [binarization]
[arxiv] Training Competitive Binary Neural Networks from Scratch. [binarization] [code] [192:star:]
[arxiv] Joint Neural Architecture Search and Quantization. [low-bit] [torch]
[CVPR] Explicit loss-error-aware quantization for low-bit deep neural networks. [low-bit]

2017

[CoRR] BMXNet: An Open-Source Binary Neural Network Implementation Based on MXNet. [binarization] [code]
[CVPR] [251:fire:] Deep Learning with Low Precision by Half-wave Gaussian Quantization. [low-bit] [code] [118:star:]
[CVPR] [156:fire:] Local Binary Convolutional Neural Networks. [binarization] [torch] [94:star:]
[FPGA] [463:fire:] FINN: A Framework for Fast, Scalable Binarized Neural Network Inference. [hardware] [binarization]
[ICASSP)] Fixed-point optimization of deep neural networks with adaptive step size retraining. [low-bit]
[ICCV] [130:fire:] Binarized Convolutional Landmark Localizers for Human Pose Estimation and Face Alignment with Limited Resources. [binarization] [homepage] [torch] [207:star:]
[ICCV] [55:fire:] Performance Guaranteed Network Acceleration via High-Order Residual Quantization. [low-bit]
[ICLR] [554:fire:] Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights. [low-bit] [torch] [144:star:]
[ICLR] [119:fire:] Loss-aware Binarization of Deep Networks. [binarization] [code]
[ICLR] [222:fire:] Soft Weight-Sharing for Neural Network Compression. [other]
[ICLR] [637:fire:] Trained Ternary Quantization. [low-bit] [torch] [90:star:]
[InterSpeech] Binary Deep Neural Networks for Speech Recognition. [binarization]
[IPDPSW] On-Chip Memory Based Binarized Convolutional Deep Neural Network Applying Batch Normalization Free Technique on an FPGA. [hardware]
[JETC] A GPU-Outperforming FPGA Accelerator Architecture for Binary Convolutional Neural Networks. [hardware] [binarization]
[NeurIPS] [293:fire:] Towards Accurate Binary Convolutional Neural Network. [binarization] [tensorflow]
[Neurocomputing] [126:fire:] FP-BNN: Binarized neural network on FPGA. [hardware]
[MWSCAS] Deep learning binary neural network on an FPGA. [hardware] [binarization]
[arxiv] [71:fire:] Ternary Neural Networks with Fine-Grained Quantization. [low-bit]
[arxiv] ShiftCNN: Generalized Low-Precision Architecture for Inference of Convolutional Neural Networks. [low-bit] [code] [53:star:]

2016

[CoRR] [1k:fire:] DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients. [low-bit] [code] [5.8k:star:]
[ECCV] [2.7k:fire:] XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks. [binarization] [torch] [787:star:]
[ICASSP)] Fixed-point Performance Analysis of Recurrent Neural Networks. [low-bit]
[NeurIPS] [572:fire:] Ternary weight networks. [low-bit] [code] [61:star:]
[NeurIPS)] [1.7k:fire:] Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1. [binarization] [torch] [239:star:]
[CVPR] [270:fire:] Quantized convolutional neural networks for mobile devices. code

2015

[ICML] [191:fire:] Bitwise Neural Networks. [binarization]
[NeurIPS] [1.8k:fire:] BinaryConnect: Training Deep Neural Networks with binary weights during propagations. [binarization] [code] [330:star:]
[arxiv] Resiliency of Deep Neural Networks under quantizations. [low-bit]

Codes_and_Docs

[code] [doc] ZF-Net: An Open Source FPGA CNN Library.
[doc] Accelerating CNN inference on FPGAs: A Survey.
[code] Different quantization methods implement by Pytorch.
[中文] Quantization Methods.
[中文] Run BNN in FPGA.
[中文] An Overview of Deep Compression Approaches.
[中文] 嵌入式深度学习之神经网络二值化（3）- FPGA实现

Our_Team

Our team is part of the DIG group of the State Key Laboratory of Software Development Environment (SKLSDE), supervised Prof. Xianglong Liu. The main research goals of our team is compressing and accelerating models under multiple scenes.

Members

Haotong Qin

Haotong Qin is a Ph.D. student in the State Key Laboratory of Software Development Environment (SKLSDE) and Shen Yuan Honors College at Beihang University, supervised by Prof. Wei Li and Prof. Xianglong Liu. And He is also a student researcher in Bytedance AI Lab. He obtained a B.Eng degree in computer science and engineering from Beihang University, and interned at the MSRA and Tencent WXG. I'm interested in hardware-friendly deep learning and neural network quantization. And his research goal is to enable state-of-the-art neural network models to be deployed on resource-limited hardware, which includes the compression and acceleration for multiple architectures, and the flexible and efficient deployment on multiple hardware.

Ruihao Gong

Ruihao Gong is currently a Ph.D. student in in the State Key Laboratory of Software Development Environment (SKLSDE) and a senior researcher at SenseTime. Since 2017, he worked on the build-up of computer vision systems and model quantization as an intern at Sensetime Research, where he enjoyed working with the talented researchers and grew up a lot with the help of Fengwei Yu, Wei Wu, Jing Shao, and Junjie Yan. During the early time of the internship, he independently took responsibility for the development of intelligent video analysis system Sensevideo. Later, he started the research on model quantization which can speed up the inference and even the training of neural networks on edge devices. Now he is devoted to further promoting the accuracy of extremely low-bit models and the auto-deployment of quantized models.

Yifu Ding

Yifu Ding is a senior student in the School of Computer Science and Engineering at Beihang University. She is in the State Key Laboratory of Software Development Environment (SKLSDE), under the supervision of Prof. Xianglong Liu. Currently, she is interested in computer vision and model quantization. She thinks that neural network models which are highly compressed can be deployed on resource-constrained devices. And among all the compression methods, quantization is a potential one.

Xiuying Wei

Xiuying Wei is a first-year graduate student at Beihang University under the supervision of Prof. Xianglong Liu. She recevied a bachelor’s degree from Shandong University in 2020. Currently, she is interested in model quantization. She thinks that quantization could make model faster and more robust, which could put deep learning systems on low-power devices and bring more opportunity for future.

Qinghua Yan

I am a senior student in the Sino-French Engineer School at Beihang University. I just started the research on model compression in the Skate Key Laboratory of Software Development Environment (SKLSDE), under the supervision of Prof. Xianglong Liu. I have great enthusiasm for deep learning and model quantization and I really enjoy working with my talented teammates.

Hong Chen

Interns

Aoyu Li

I am a senior student in the School of Computer Science and Engineering at Beihang University. Supervised by Prof. Xianglong Liu, I am currently conducting research on model quantization. My research interests are mainly about model compression, deployment and AI systems.

Xudong Ma

Yuxuan Wen

Alumnus

Xiangguo Zhang

Xiangguo Zhang achieved the master degree in the School of Computer Science of Beihang University, under the guidance of Prof. Xianglong Liu. He received a bachelor's degree from Shandong University in 2019 and entered Beihang University in the same year. Currently, he is interested in computer vision and post training quantization.

Mingzhu Shen

Publications

Xiangguo Zhang*, Haotong Qin*, Yifu Ding, Ruihao Gong, Qinghua Yan, Renshuai Tao, Yuhang Li, Fengwei Yu, Xianglong Liu. Diversifying Sample Generation for Data-Free Quantization. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021 (Oral). (* equal contribution)
Haotong Qin*, Zhongang Cai*, Mingyuan Zhang*, Yifu Ding, Haiyu Zhao, Shuai Yi, Xianglong Liu, Hao Su. BiPointNet: Binary Neural Network for Point Clouds. International Conference on Learning Representations (ICLR), 2021. (* equal contribution)

Haotong Qin, Ruihao Gong, Xianglong Liu, Xiao Bai, Jingkuan Song, Nicu Sebe. Binary Neural Network: A Survey. Pattern Recognition (PR), 2020.
Haotong Qin, Ruihao Gong, Xianglong Liu, Mingzhu Shen, Ziran Wei, Fengwei Yu, Jingkuan Song. Forward and Backward Information Retention for Accurate Binary Neural Networks. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Y. Wu, X. Liu*, H. Qin , K. Xia, S. Hu, Y. Ma, M. Wang. Boosting Temporal Binary Coding for Large-scale Video Search. IEEE Transactions on Multimedia (TMM), 2020.
F. Zhu, R. Gong, F. Yu, X. Liu, Y. Wang, Z. Li, X. Yang, J. Yan. Towards Unified INT8 Training for Convolutional Neural Network. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Y. Li, R. Gong, F. Yu, X. Dong, X. Liu. DMS: Differentiable Dimension Search for Binary Neural Networks, International Conference on Learning Representations workshop on Neural Architecture Search (ICLR NAS workshop), 2020.
Y. Wu, Y. Wu, R. Gong, Y. Lv, K. Chen, D. Liang, X. Hu, X. Liu, J. Yan. Rotation Consistent Margin Loss for Efficient Low-bit Face Recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
M. Shen, X. Liu, R. Gong, K. Han. Balanced Binary Neural Networks with Gated Residual. IEEE International Conference on Acoustics, Speech, and Signal Processing, 2020.
R. Gong, X. Liu*, S. Jiang, T. Li, P. Hu, J. Lin, F. Yu, J. Yan. Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks. IEEE International Conference on Computer Vision (CVPR), 2019.

MinglongLei/awesome-model-quantization

Awesome Model Quantization

Table of Contents

Benchmark

MQBench

Survey_Papers

Survey_of_Binarization

Survey_of_Quantization

Papers

2021

2020

2019

2018

2017

2016

2015

Codes_and_Docs

Our_Team

Members

Interns

Alumnus

Publications