pvti/Awasome-Pruning

Awasome Papers and Resources in Deep Neural Network Pruning with Source Code.

Awesome Pruning

Awesome resources in deep neural network pruning. This collection is inspired by he-y/Awesome-Pruning.

[Note: You are welcome to create pool requests and add more interesting papers.]

Section	Year of Publication
Conference Publications	2024 2023 2022 2021 2020 2019 2018 2017
Journal Publications	2024 2023 2022 2021 2020
Survey Articles	2020~2023
Other Publications	2022~2023
Pruning Software and Toolbox	2019~2023

Symbol	Meaning
U	Unstructured or Weight Pruning
S	Structured or Filter or Channel Pruning
A	Official or Author Implementation
O	Unofficial or 3rd Party Implementation

Conference Publications

2024

Venue	Title	Type	Code
ICLR	Towards Meta-Pruning via Optimal Transport	S	PyTorch[A]
ICLR	Towards Energy Efficient Spiking Neural Networks: An Unstructured Pruning Framework	U	PyTorch[A]
ICLR	Masks, Signs, And Learning Rate Rewinding	S	PyTorch[A]
ICLR	Scaling Laws for Sparsely-Connected Foundation Models	S	PyTorch[A]
ICLR	Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging	S
ICLR	Adaptive Sharpness-Aware Pruning for Robust Sparse Networks	S
ICLR	What Makes a Good Prune? Maximal Unstructured Pruning for Maximal Cosine Similarity	U	PyTorch[A]
ICLR	In defense of parameter sharing for model-compression	S/U
ICLR	ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Models	U
ICLR	Data-independent Module-aware Pruning for Hierarchical Vision Transformers	S	PyTorch[A]
ICLR	SWAP: Sparse Entropic Wasserstein Regression for Robust Network Pruning	S
ICLR	Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning	U
ICLR	Synergistic Patch Pruning for Vision Transformer: Unifying Intra- & Inter-Layer Patch Importance	S
ICLR	FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity	S
ICLR	The Need for Speed: Pruning Transformers with One Recipe	S	PyTorch[A]
ICLR	SAS: Structured Activation Sparsification	S	PyTorch[A]
CVPR	OrthCaps: An Orthogonal CapsNet with Sparse Attention Routing and Pruning	S	PyTorch[A]
CVPR	Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformers	S	PyTorch[A]
CVPR	Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruning	S	PyTorch[A]
CVPR	BilevelPruning: Unified Dynamic and Static Channel Pruning for Convolutional Neural Networks	S
CVPR	FedMef: Towards Memory-efficient Federated Dynamic Pruning	S
CVPR	Resource-Efficient Transformer Pruning for Finetuning of Large Models	S
CVPR	Device-Wise Federated Network Pruning	S
CVPR	Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch	S
CVPR	Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignment	S
CVPR	Diversity-aware Channel Pruning for StyleGAN Compression	S	PyTorch[A]
CVPR	MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformer	S	PyTorch[A]
AAAI	Dynamic Feature Pruning and Consolidation for Occluded Person Re-Identification	S
AAAI	REPrune: Channel Pruning via Kernel Representative Selection	S
AAAI	Revisiting Gradient Pruning: A Dual Realization for Defending against Gradient Attacks	S
AAAI	IRPruneDet: Efficient Infrared Small Target Detection via Wavelet Structure-Regularized Soft Channel Pruning	S
AAAI	EPSD: Early Pruning with Self-Distillation for Efficient Model Compression	S
WACV	Pruning from Scratch via Shared Pruning Module and Nuclear norm-based Regularization	S	PyTorch[A]
WACV	Towards Better Structured Pruning Saliency by Reorganizing Convolution	S	PyTorch[A]
WACV	Torque based Structured Pruning for Deep Neural Network	S
WACV	Revisiting Token Pruning for Object Detection and Instance Segmentation	S	PyTorch[A]
WACV	Token Fusion: Bridging the Gap Between Token Pruning and Token Merging	S
WACV	PATROL: Privacy-Oriented Pruning for Collaborative Inference Against Model Inversion Attacks	S

2023

Venue	Title	Type	Code
NIPS	Diff-Pruning: Structural Pruning for Diffusion Models	S	PyTorch[A]
NIPS	LLM-Pruner: On the Structural Pruning of Large Language Models	S	PyTorch[A]
ICCV	Automatic Network Pruning via Hilbert-Schmidt Independence Criterion Lasso under Information Bottleneck Principle	S	PyTorch[A]
ICCV	Unified Data-Free Compression: Pruning and Quantization without Fine-Tuning	S	PyTorch[A]
ICCV	Structural Alignment for Network Pruning through Partial Regularization	S
ICCV	Differentiable Transportation Pruning	S
ICCV	Dynamic Token Pruning in Plain Vision Transformers for Semantic Segmentation	S	PyTorch[A]
ICCV	Towards Fairness-aware Adversarial Network Pruning	S
ICCV	Efficient Joint Optimization of Layer-Adaptive Weight Pruning in Deep Neural Networks	S	PyTorch[A]
CVPR	DepGraph: Towards Any Structural Pruning	S	PyTorch[A]
CVPR	X-Pruner: eXplainable Pruning for Vision Transformers	U/S
CVPR	Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers	S	PyTorch[A]
CVPR	Global Vision Transformer Pruning with Hessian-Aware Saliency	S
CVPR	CP3: Channel Pruning Plug-in for Point-based Networks	S
CVPR	Training Debiased Subnetworks With Contrastive Weight Pruning	U
CVPR	Pruning Parameterization With Bi-Level Optimization for Efficient Semantic Segmentation on the Edge	S
CVPR	Structural Alignment for Network Pruning through Partial Regularization	S	PyTorch[A]
ICLR	JaxPruner: A concise library for sparsity research	U/S	PyTorch[A]
ICLR	OTOv2: Automatic, Generic, User-Friendly	S	PyTorch[A]
ICLR	How I Learned to Stop Worrying and Love Retraining	U	PyTorch[A]
ICLR	Token Merging: Your ViT But Faster	U/S	PyTorch[A]
ICLR	Revisiting Pruning at Initialization Through the Lens of Ramanujan Graphs	U	PyTorch[A] (soon...)
ICLR	Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?	U
ICLR	NTK-SAP: Improving neural network pruning by aligning training dynamics	U
ICLR	DFPC: Data flow driven pruning of coupled channels without data	S	PyTorch[A]
ICLR	TVSPrune - Pruning Non-discriminative filters via Total Variation separability of intermediate representations without fine tuning	S	PyTorch[A]
ICLR	Pruning Deep Neural Networks from a Sparsity Perspective	U	PyTorch[A]
ICLR	A Unified Framework of Soft Threshold Pruning	U	PyTorch[A]
WACV	Calibrating Deep Neural Networks Using Explicit Regularisation and Dynamic Data Pruning	S
WACV	Attend Who Is Weak: Pruning-Assisted Medical Image Localization Under Sophisticated and Implicit Imbalances	S
ICASSP	WHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networks	S	PyTorch[A]

2022

Venue	Title	Type	Code
CVPR	Interspace Pruning: Using Adaptive Filter Representations To Improve Training of Sparse CNNs	U
CVPR	Revisiting Random Channel Pruning for Neural Network Compression	S	PyTorch[A] (soon...)
CVPR	Fire Together Wire Together: A Dynamic Pruning Approach With Self-Supervised Mask Prediction	S	PyTorch[A]
CVPR	When to Prune? A Policy towards Early Structural Pruning	S
CVPR	Dreaming to Prune Image Deraining Networks	S
ICLR	SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruning	S
ICLR	Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining	U	PyTorch[A]
ICLR	Revisit Kernel Pruning with Lottery Regulated Grouped Convolutions	S	PyTorch[A]
ICLR	Dual Lottery Ticket Hypothesis	U	PyTorch[A]
NIPS	SAViT: Structure-Aware Vision Transformer Pruning via Collaborative Optimization	S	PyTorch[A](soon...)
NIPS	Structural Pruning via Latency-Saliency Knapsack	S	PyTorch[A]
ACCV	Filter Pruning via Automatic Pruning Rate Search⋆	S
ACCV	Network Pruning via Feature Shift Minimization	S	PyTorch[A]
ACCV	Lightweight Alpha Matting Network Using Distillation-Based Channel Pruning	S	PyTorch[A]
ACCV	Adaptive FSP : Adaptive Architecture Search with Filter Shape Pruning	S
ECCV	Soft Masking for Cost-Constrained Channel Pruning	S	PyTorch[A]
WACV	Hessian-Aware Pruning and Optimal Neural Implant	S	PyTorch[A]
WACV	PPCD-GAN: Progressive Pruning and Class-Aware Distillation for Large-Scale Conditional GANs Compression	S
WACV	Channel Pruning via Lookahead Search Guided Reinforcement Learning	S
WACV	EZCrop: Energy-Zoned Channels for Robust Output Pruning	S	PyTorch[A]
ICIP	One-Cycle Pruning: Pruning Convnets With Tight Training Budget	U
ICIP	RAPID: A Single Stage Pruning Framework	U
ICIP	The Rise of the Lottery Heroes: Why Zero-Shot Pruning is Hard	U
ICIP	Truncated Lottery Ticket for Deep Pruning	U
ICIP	Which Metrics For Network Pruning: Final Accuracy? or Accuracy Drop?	S/U
ISMSI	Structured Pruning with Automatic Pruning Rate Derivation for Image Processing Neural Networks	S

2021

Venue	Title	Type	Code
ICLR	Neural Pruning via Growing Regularization	S	PyTorch[A]
ICLR	Network Pruning That Matters: A Case Study on Retraining Variants	S	PyTorch[A]
ICLR	Layer-adaptive Sparsity for the Magnitude-based Pruning	U	PyTorch[A]
NIPS	Only Train Once: A One-Shot Neural Network Training And Pruning Framework	S	PyTorch[A]
CVPR	NPAS: A Compiler-Aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration	S
CVPR	Network Pruning via Performance Maximization	S
CVPR	Convolutional Neural Network Pruning With Structural Redundancy Reduction*	S
CVPR	Manifold Regularized Dynamic Network Pruning	S	PyTorch[A]
CVPR	Joint-DetNAS: Upgrade Your Detector With NAS, Pruning and Dynamic Distillation	S
ICCV	ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting	S
ICCV	Achieving On-Mobile Real-Time Super-Resolution With Neural Architecture and Pruning Search	S
ICCV	GDP: Stabilized Neural Network Pruning via Gates With Differentiable Polarization*	S
WACV	Holistic Filter Pruning for Efficient Deep Neural Networks	S
ICML	Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Framework	S
ICML	Group Fisher Pruning for Practical Network Compression	S	PyTorch[A]

2020

Venue	Title	Type	Code
CVPR	HRank: Filter Pruning using High-Rank Feature Map	S	PyTorch[A]
CVPR	Towards efficient model compression via learned global ranking	S	PyTorch[A]
CVPR	Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration	S
CVPR	Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression	S	PyTorch[A]
CVPR	APQ: Joint Search for Network Architecture, Pruning and Quantization Policy	S	PyTorch[A]
ICLR	Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraints	U
MLSys	Shrinkbench: What is the State of Neural Network Pruning?		PyTorch[A]
BMBS	Similarity Based Filter Pruning for Efficient Super-Resolution Models	S

2019

Venue	Title	Type	Code
CVPR	Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration	S	PyTorch[A]
CVPR	Variational Convolutional Neural Network Pruning	S
CVPR	Towards Optimal Structured CNN Pruning via Generative Adversarial Learning	S	PyTorch[A]
CVPR	Partial Order Pruning: For Best Speed/Accuracy Trade-Off in Neural Architecture Search	S	PyTorch[A]
CVPR	Importance Estimation for Neural Network Pruning	S	PyTorch[A]
ICLR	The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks	U	PyTorch[A]
ICLR	SNIP: Single-shot Network Pruning based on Connection Sensitivity	U	Tensorflow[A]
ICCV	MetaPruning: Meta-Learning for Automatic Neural Network Channel Pruning	S	PyTorch[A]
ICCV	Accelerate CNN via Recursive Bayesian Pruning	S

2018

Venue	Title	Type	Code
CVPR	PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning	S	PyTorch[A]
CVPR	NISP: Pruning Networks Using Neuron Importance Score Propagation	S
ICIP	Online Filter Clustering and Pruning for Efficient Convnets	S
IJCAI	Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks	S	PyTorch[A]

2017

Venue	Title	Type	Code
CVPR	Designing Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning	S
ICLR	Pruning Filters for Efficient ConvNets	S	PyTorch[O]
ICCV	Channel Pruning for Accelerating Very Deep Neural Networks	S	PyTorch[A]
ICCV	ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression	S	Caffe[A]
ICCV	Learning Efficient Convolutional Networks Through Network Slimming	S	PyTorch[A]

Journal Publications

2024

Journal	Title	Type	Code
Neural Networks	Efficient tensor decomposition-based filter pruning	S	PyTorch[A]
IEEE Trans. NNLS	Enhanced Network Compression Through Tensor Decompositions and Pruning	S	PyTorch[A]
IEEE Transactions on Artificial Intelligence	Distilled Gradual Pruning with Pruned Fine-tuning	U	PyTorch[A]

2023

Journal	Title	Type	Code
IEEE Trans Circuits Syst Video Technol	DCFP: Distribution Calibrated Filter Pruning for Lightweight and Accurate Long-tail Semantic Segmentation	S
IEEE Internet Things J.	SNPF: Sensitiveness Based Network Pruning Framework for Efficient Edge Computing	S
IEEE Trans. NNLS	Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN	S
IEEE Trans. NNLS	Block-Wise Partner Learning for Model Compression	S	PyTorch[A]
IEEE Trans. NNLS	Hierarchical Threshold Pruning Based on Uniform Response Criterion	S
IEEE Trans. NNLS	CATRO: Channel Pruning via Class-Aware Trace Ratio Optimization	S
IEEE Trans. NNLS	Adaptive Filter Pruning via Sensitivity Feedback	S
Neurocomputing	Filter pruning with uniqueness mechanism in the frequency domain for efficient neural networks	S
IEEE Trans. PAMI	Compact Neural Network via Stacking Hybrid Units	S
IEEE Trans. PAMI	Performance-aware Approximation of Global Channel Pruning for Multitask CNNs	S	PyTorch[A]
IEEE Trans. PAMI	Adaptive Search-and-Training for Robust and Efficient Network Pruning	S
Image Vis. Comput.	Loss-aware automatic selection of structured pruning criteria for deep neural network acceleration	S	PyTorch[A]
Comput. Vis. Image Underst.	Feature independent Filter Pruning by Successive Layers analysis	S
IEEE Access	Differentiable Neural Architecture, Mixed Precision and Accelerator Co-Search	S

2022

Journal	Title	Type	Code
IEEE Trans. Image Process.	Efficient Layer Compression Without Pruning	S
IEEE Trans. PAMI	Learning to Explore Distillability and Sparsability: A Joint Framework for Model Compression	S
IEEE Trans. PAMI	1xN Pattern for Pruning Convolutional Neural Networks	S	PyTorch[A]
IEEE Trans. NNLS	Filter Pruning by Switching to Neighboring CNNs With Good Attribute	S
IEEE Trans. NNLS	Model Pruning Enables Efficient Federated Learning on Edge Devices	S
IEEE Trans. NNLS	DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Search	S
IEEE Trans. NNLS	Network Pruning Using Adaptive Exemplar Filters	S	PyTorch[A]
IEEE Trans. NNLS	Carrying Out CNN Channel Pruning in a White Box	S	PyTorch[A]
IEEE Trans. NNLS	Pruning Networks With Cross-Layer Ranking & k-Reciprocal Nearest Filters	S	PyTorch[A]
IEEE Trans. NNLS	Filter Sketch for Network Pruning	S	PyTorch[A]
Neurocomputing	FPFS: Filter-level pruning via distance weight measuring filter similarity	S
Neurocomputing	RUFP: Reinitializing unimportant filters for soft pruning	S
Neural Netw	HRel: Filter pruning based on High Relevance between activation maps and class labels	S	PyTorch[A]*
Comput. Intell. Neurosci.	Differentiable Network Pruning via Polarization of Probabilistic Channelwise Soft Masks	S
J. Syst. Archit.	Optimizing deep neural networks on intelligent edge accelerators via flexible-rate filter pruning	S
Appl. Sci.	Magnitude and Similarity Based Variable Rate Filter Pruning for Efficient Convolution Neural Networks	S	PyTorch[A]
Sensors	Filter Pruning via Measuring Feature Map Information	S
IEEE Access	Automated Filter Pruning Based on High-Dimensional Bayesian Optimization	S
IEEE Signal Process. Lett.	A Low-Complexity Modified ThiNet Algorithm for Pruning Convolutional Neural Networks	S

2021

Journal	Title	Type	Code
IEEE Trans. PAMI	Discrimination-Aware Network Pruning for Deep Model Compression	S	PyTorch[A]~

2020

Journal	Title	Type	Code
IEEE Trans. NNLS	EDP: An Efficient Decomposition and Pruning Scheme for Convolutional Neural Network Compression	S
IEEE Access	Filter Pruning Without Damaging Networks Capacity	S
Electronics	Pruning Convolutional Neural Networks with an Attention Mechanism for Remote Sensing Image Classification	S

Survey Articles

Year	Venue	Title
2023	Artif. Intell. Rev.	Deep neural network pruning method based on sensitive layers and reinforcement learning
2023	arVix	A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations
2023	arVix	Structured Pruning for Deep Convolutional Neural Networks: A survey
2022	Electronics	A Survey on Efficient Convolutional Neural Networks and Hardware Acceleration
2022	I-SMAC	A Survey on Filter Pruning Techniques for Optimization of Deep Neural Networks
2021	JMLR	Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
2021	Neurocomputing	Pruning and quantization for deep neural network acceleration: A survey
2020	IEEE Access	Methods for Pruning Deep Neural Networks

Other Publications

Year	Venue	Title	Code
2023	arVix	Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning	PyTorch[A](soon...)
2023	arVix	Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researchers
2022	ICML	Tutorial -- Sparsity in Deep Learning: Pruning and growth for efficient inference and training

Pruning Software and Toolbox

Year	Title	Type	Code
2023	UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers	S	PyTorch[A]
2023	DepGraph: Towards Any Structural Pruning	S	PyTorch[A]
2023	Torch-Pruning	S	PyTorch[A]
2023	JaxPruner: JaxPruner: A concise library for sparsity research	U/S	PyTorch[A]
2022	FasterAI: Prune and Distill your models with FastAI and PyTorch	U	PyTorch[A]
2022	Simplify: A Python library for optimizing pruned neural networks		PyTorch[A]
2021	PyTorchViz [A small package to create visualizations of PyTorch execution graphs]		PyTorch[A]
2020	What is the State of Neural Network Pruning?	S/U	PyTorch[A]
2019	Official PyTorch Pruning Tool	S/U	PyTorch[A]