/Four-Lightweight-Methods

The four simple lightweight methods examples. Including Model Pruning, Model Quantization, Knowledge Distillation and Compact Model.

Primary LanguagePython

Four Simple Lightweight Methods Examples

This project is the PyTorch implementation

Dependencies:

  • Python == 3.8
  • Pytorch >= 1.9

Sub-item:

  • Model Pruning
  • Model Quantization
  • Knowledge Distillation
  • Compact Model

Suggested reading order: Model Pruning, Model Quantization, Knowledge Distillation, Compact Models

Wzixin

FFFFFFFFFFFFFFFFFFFFFF                                                        LLLLLLLLLLL     WWWWWWWW                           WWWWWWWW
F::::::::::::::::::::F                                                        L:::::::::L     W::::::W                           W::::::W
F::::::::::::::::::::F                                                        L:::::::::L     W::::::W                           W::::::W
FF::::::FFFFFFFFF::::F                                                        LL:::::::LL     W::::::W                           W::::::W
  F:::::F       FFFFFFooooooooooo   uuuuuu    uuuuuu rrrrr   rrrrrrrrr          L:::::L        W:::::W           WWWWW           W:::::W 
  F:::::F           oo:::::::::::oo u::::u    u::::u r::::rrr:::::::::r         L:::::L         W:::::W         W:::::W         W:::::W  
  F::::::FFFFFFFFFFo:::::::::::::::ou::::u    u::::u r:::::::::::::::::r        L:::::L          W:::::W       W:::::::W       W:::::W   
  F:::::::::::::::Fo:::::ooooo:::::ou::::u    u::::u rr::::::rrrrr::::::r       L:::::L           W:::::W     W:::::::::W     W:::::W    
  F:::::::::::::::Fo::::o     o::::ou::::u    u::::u  r:::::r     r:::::r       L:::::L            W:::::W   W:::::W:::::W   W:::::W     
  F::::::FFFFFFFFFFo::::o     o::::ou::::u    u::::u  r:::::r     rrrrrrr       L:::::L             W:::::W W:::::W W:::::W W:::::W      
  F:::::F          o::::o     o::::ou::::u    u::::u  r:::::r                   L:::::L              W:::::W:::::W   W:::::W:::::W       
  F:::::F          o::::o     o::::ou:::::uuuu:::::u  r:::::r                   L:::::L         LLLLLLW:::::::::W     W:::::::::W        
FF:::::::FF        o:::::ooooo:::::ou:::::::::::::::uur:::::r                 LL:::::::LLLLLLLLL:::::L W:::::::W       W:::::::W         
F::::::::FF        o:::::::::::::::o u:::::::::::::::ur:::::r                 L::::::::::::::::::::::L  W:::::W         W:::::W          
F::::::::FF         oo:::::::::::oo   uu::::::::uu:::ur:::::r                 L::::::::::::::::::::::L   W:::W           W:::W           
FFFFFFFFFFF           ooooooooooo       uuuuuuuu  uuuurrrrrrr                 LLLLLLLLLLLLLLLLLLLLLLLL    WWW             WWW