pdh930105/awesome-reparameterize

Awesome-Reparameterize

This page's style is based on Awesome-Efficient-LLM. Thank you for providing a great format.

A curated list for Reparameterize:

• CNN
• Transformer/ViT
• Quantization for Reparameterize
• Specific Domain
• Others

🚀 Future Works

[ ] Sorting papers by year, conference

[ ] Upload an image of the main idea of the paper

[] Support journal format (TPAMI, TNNLS, TIP, etc)

💮 Contributing

If you'd like to include other reparameterization paper, or need to update any details such as conference information or code URLs, please feel free to submit a pull request. You can generate the required markdown format for each paper by filling in the information in generate_item.py and execute python generate_item.py. (or using genetarte_markdown.ipynb) We warmly appreciate your contributions to this list. Alternatively, you can email me with the links to your paper and code, and I would add your paper to the list at my earliest convenience.

CNN

Title & Authors	Introduction	Links
ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks Xiaohan. Ding, Yuchen. Guo, Guiguang. Ding, Jungong. Han,		Github Paper
Diverse branch block: Building a convolution as an inception-like unit Xiaohan. Ding, Xiangyu. Zhang, Jungong. Han, Guiguang. Ding,		Github Paper
Repvgg: Making vgg-style convnets great again Xiaohan. Ding, Xiangyu. Zhang, Ningning. Ma, Jungong. Han, Guiguang. Ding, Jian. Sun,		Github Paper
MobileOne: An Improved One Millisecond Mobile Backbone Pavan. Vasu, James. Gabriel, Jeff. Zhu, Oncel. Tuzel, Anurag. Ranjan,		Github Paper
Scaling up your kernels to 31x31: Revisiting large kernel design in cnns Xiaohan. Ding, Xiangyu. Zhang, Jungong. Han, Guiguang. Ding,		Github Paper
UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition Xiaohan. Ding, Yiyuan. Zhang, Yixiao. Ge, Sijie. Zhao, Lin. Song, Xiangyu. Yue, Ying. Shan,		Github Paper
More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity Shiwei. Liu, Tianlong. Chen, Xiaohan. Chen, Xuxi. Chen, Qiao. Xiao, Boqian. Wu, Tommi. K{"a}rkk{"a}inen, Mykola. Pechenizkiy, Decebal. Mocanu, Zhangyang. Wang,		Github Paper
Repghost: a hardware-efficient ghost module via re-parameterization Chengpeng. Chen, Zichao. Guo, Haien. Zeng, Pengfei. Xiong, Jian. Dong,		Github Paper
RepRCNN: A Structural Reparameterisation Convolutional Neural Network Object Detection Algorithm Based on Branch Matching Xudong. Li, Xinyao. Lv, Linghui. Sun, Jingzhi. Zhang, Ruoming. Lan,		Paper
DepthShrinker: a new compression paradigm towards boosting real-hardware efficiency of compact neural networks Yonggan. Fu, Haichuan. Yang, Jiayi. Yuan, Meng. Li, Cheng. Wan, Raghuraman. Krishnamoorthi, Vikas. Chandra, Yingyan. Lin,		Github Paper
NetBooster: Empowering Tiny Deep Learning By Standing on the Shoulders of Deep Giants Zhongzhi. Yu, Yonggan. Fu, Jiayi. Yuan, Haoran. You, Yingyan. Lin,		Paper
Dyrep: bootstrapping training with dynamic re-parameterization Tao. Huang, Shan. You, Bohan. Zhang, Yuxuan. Du, Fei. Wang, Chen. Qian, Chang. Xu,		Github Paper
ASR: Attention-alike Structural Re-parameterization Shanshan. Zhong, Zhongzhan. Huang, Wushao. Wen, Jinghui. Qin, Liang. Lin,		Paper
Efficient Latency-Aware CNN Depth Compression via Two-Stage Dynamic Programming Jinuk. Kim, Yeonwoo. Jeong, Deokjae. Lee, Hyun. Song,		Github Paper

Transformer/ViT

Title & Authors	Introduction	Links
Repq-vit: Scale reparameterization for post-training quantization of vision transformers Zhikai. Li, Junrui. Xiao, Lianwei. Yang, Qingyi. Gu,		Github Paper
FastViT: A Fast Hybrid Vision Transformer Using Structural Reparameterization Pavan. Vasu, James. Gabriel, Jeff. Zhu, Oncel. Tuzel, Anurag. Ranjan,		Github Paper
Repvit: Revisiting mobile cnn from vit perspective Ao. Wang, Hui. Chen, Zijia. Lin, Hengjun. Pu, Guiguang. Ding,		Github Paper
FMViT: A multiple-frequency mixing Vision Transformer Wei. Tan, Yifeng. Geng, Xuansong. Xie,		Github Paper

Quantization-for-Reparameterize

Title & Authors	Introduction	Links
Make RepVGG Greater Again: A Quantization-aware Approach Xiangxiang. Chu, Liang. Li, Bo. Zhang,		Paper
RepQ: Generalizing Quantization-Aware Training for Re-Parametrized Architectures Anastasiia. Prutianova, Alexey. Zaytsev, Chung-Kuei. Lee, Fengyu. Sun, Ivan. Koryakovskiy,		Paper
Post-Training Quantization for Re-parameterization via Coarse & Fine Weight Splitting Dawei. Yang, Ning. He, Xing. Hu, Zhihang. Yuan, Jiangyong. Yu, Chen. Xu, Zhe. Jiang,		Github Paper

Specific-domain (Object Detection, Segmantation, etc...)

Title & Authors	Introduction	Links
RepMode: Learning to Re-parameterize Diverse Experts for Subcellular Structure Prediction Donghao. Zhou, Chunbin. Gu, Junde. Xu, Furui. Liu, Qiong. Wang, Guangyong. Chen, Pheng-Ann. Heng,		Github Paper
RepViT-SAM: Towards Real-Time Segmenting Anything Ao. Wang, Hui. Chen, Zijia. Lin, Jungong. Han, Guiguang. Ding,		Github Paper
6d rotation representation for unconstrained head pose estimation Thorsten. Hempel, Ahmed. Abdelrahman, Ayoub. Al-Hamadi,		Github Paper
YOLOv6: A single-stage object detection framework for industrial applications Chuyi. Li, Lulu. Li, Hongliang. Jiang, Kaiheng. Weng, Yifei. Geng, Liang. Li, Zaidan. Ke, Qingyuan. Li, Meng. Cheng, Weiqiang. Nie, others,		Github Paper
YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors Chien-Yao. Wang, Alexey. Bochkovskiy, Hong-Yuan. Liao,		Github Paper
Ultralytics YOLO (YOLO v8) Glenn. Jocher, Ayush. Chaurasia, Jing. Qiu,		Github Docs

Others

Title & Authors	Introduction	Links
Re-parameterizing your optimizers rather than architectures X. Ding, H. Chen, X. Zhang, K. Huang, J. Han, G. Ding,		Github Paper
Repmlpnet: Hierarchical vision mlp with re-parameterized locality Xiaohan. Ding, Honghao. Chen, Xiangyu. Zhang, Jungong. Han, Guiguang. Ding,		Github Paper
Spatial Re-parameterization for N: M Sparsity Yuxin. Zhang, Mingbao. Lin, Yunshan. Zhong, Mengzhao. Chen, Fei. Chao, Rongrong. Ji,		Github Paper
Reparameterization through Spatial Gradient Scaling Alexander. Detkov, Mohammad. Salameh, Muhammad. Fetrat, Jialin. Zhang, Robin. Luwei, SHANGLING. JUI, Di. Niu,		Github Paper
Online convolutional re-parameterization Mu. Hu, Junyi. Feng, Jiashen. Hua, Baisheng. Lai, Jianqiang. Huang, Xiaojin. Gong, Xian-Sheng. Hua,		Github Paper
Resrep: Lossless cnn pruning via decoupling remembering and forgetting Xiaohan. Ding, Tianxiang. Hao, Jianchao. Tan, Ji. Liu, Jungong. Han, Yuchen. Guo, Guiguang. Ding,		Github Paper