/Awesome-Tensor-Decomposition

😎 A curated list of tensor decomposition resources for network compression and acceleration.

GNU General Public License v3.0GPL-3.0

Awesome Tensor Decomposition

A curated list of tensor decomposition resources for network compression.

📋 Research Papers

Title Venue Year
Compression-aware Training of Neural Networks using Frank-Wolfe GitHub Repo stars Arxiv 2024
Unified Low-rank Compression Framework for Click-through Rate Prediction GitHub Repo stars KDD2024 ADS 2024
A Practical Approach for Employing Tensor Train Decomposition in Edge Devices International Journal of Parallel Programming 2024
Structure-Preserving Network Compression Via Low-Rank Induced Training Through Linear Layers Composition Arxiv 2024
LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models GitHub Repo stars NAACL 2024
CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimization Arxiv 2024
FLoRA: Low-Rank Core Space for N-dimension GitHub Repo stars Arxiv 2024
Reduced storage direct tensor ring decomposition for convolutional neural networks compression GitHub Repo stars Arxiv 2024
Federated Learning Using Coupled Tensor Train Decomposition Arxiv 2024
Neural Network Compression Based on Tensor Ring Decomposition TNNLS 2024
Enhanced network compression through tensor decompositions and pruning GitHub Repo stars TNNLS 2024
Enhancing GAN Performance Through Neural Architecture Search and Tensor Decomposition GitHub Repo stars ICASSP 2024
Deep Convolutional Neural Network Compression Method: Tensor Ring Decomposition with Variational Bayesian Approach Neural Processing Letters 2024
Deep Learning Model Compression With Rank Reduction in Tensor Decomposition TNNLS 2023
Mixed-TD: Efficient Neural Network Accelerator with Layer-Specific Tensor Decomposition GitHub Repo stars FPL 2023
SVD-NAS: Coupling Low-Rank Approximation and Neural Architecture Search GitHub Repo stars WACV 2023
How Informative is the Approximation Error from Tensor Decomposition for Neural Network Compression? ICLR 2023
FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformer AAAI 2023
Compressing convolutional neural networks with hierarchical Tucker-2 decomposition GitHub Repo stars Applied Soft Computing 2023
Tensor shape search for efficient compression of tensorized data and neural networks Applied Soft Computing 2023
An effective low-rank compression with a joint rank selection followed by a compression-friendly training Neural Networks 2023
Joint matrix decomposition for deep convolutional neural networks compression Neurocomputing 2023
Training Acceleration of Low-Rank Decomposed Networks using Sequential Freezing and Rank Quantization Arxiv 2023
HODEC: Towards Efficient High-Order DEcomposed Convolutional Neural Networks CVPR 2022
Convolutional Neural Network Compression through Generalized Kronecker Product Decomposition AAAI 2022
Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination GitHub Repo stars SIMODS 2022
Deep neural network compression by Tucker decomposition with nonlinear response Knowledge-Based Systems 2022
Nested compression of convolutional neural networks with Tucker-2 decomposition IJCNN 2022
A Design Space Exploration Methodology for Enabling Tensor Train Decomposition in Edge Devices SAMOS 2022
Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition GitHub Repo stars NeurIPS 2021
Deeply Shared Filter Bases for Parameter-Efficient Convolutional Neural Networks GitHub Repo stars NeurIPS 2021
Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Framework CVPR 2021
Deep Convolutional Neural Network Compression via Coupled Tensor Decomposition JSTSP 2021
Low-Rank Compression of Neural Nets: Learning the Rank of Each Layer GitHub Repo stars CVPR 2020
Learning Low-rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsification GitHub Repo stars CVPRW 2020
ADA-Tucker: Compressing deep neural networks via adaptive dimension adjustment tucker decomposition Neural Networks 2019
Learning Filter Basis for Convolutional Neural Network Compression GitHub Repo stars ICCV 2019
Compressing Deep Models using Multi Tensor Train Decomposition ICCAIS 2019
Compressing Fully Connected Layers using Kronecker Tensor Decomposition ICCSNT 2019
Adaptive Mixture of Low-Rank Factorizations for Compact Neural Modeling GitHub Repo stars OpenReview 2019
On Compressing Deep Models by Low Rank and Sparse Decomposition ECCV 2018
On Compressing Deep Models by Low Rank and Sparse Decomposition CVPR 2017
Factorized Convolutional Neural Networks ICCVW 2017
Accelerating Very Deep Convolutional Networks for Classification and Detection TPAMI 2016
Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications ICLR 2016
Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition ICLR 2015
Speeding up Convolutional Neural Networks with Low Rank Expansions Arxiv 2014

📚 Surveys

Title Venue Year
Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Training Arxiv 2023
Tensor Networks Meet Neural Networks: A Survey and Future Perspectives GitHub Repo stars Arxiv 2023
Tensor Decomposition for Signal Processing and Machine Learning IEEE Transactions on Signal Processing 2017
The Higher-Order Singular Value Decomposition: Theory and an Application IEEE Signal Processing Magazine 2010
Tensor Decompositions and Applications SIAM Review 2009

📘 Miscellaneous

Title Venue Year
Nuclear Norm of Higher-Order Tensors Mathematics of Computation 2018

💻 Repositories