/Model-Compression-Keras

cnn compression for keras

Primary LanguagePythonMIT LicenseMIT

Model-Compression-Keras

cnn compression for keras

English is not my native language; please excuse typing errors.

The implementation for "deep compression"(only apply weight pruning and quantization).

This implementation is a little different from original mathod in the paper.

Pruning small weights for each convolution filter. The pruning threshold is set by handcraft.

How to use

  1. Train the model (this repo currently support normal-cnn, ResNet, Inception, Densenet, Unet, PointNet)

For CIFAR-10 and compression rate=0.8:

python train_and_compress_cnn.py --model='vgg' --data='c10' --compress-rate=0.8

For CIFAR-100 and compression rate=0.8:

python train_and_compress_cnn.py --model='vgg' --data='c100' --compress-rate=0.8

For UNet and compression rate=0.8:

python train_and_compress_unet.py --compress-rate=0.8

For PointNet and compression rate=0.8:

python train_and_compress_pointnet.py --compress-rate=0.8

  1. Decode and evaluation:

    python decode_and_evaluate_cnn.py --model='vgg'

    python decode_and_evaluate_unet.py

    python decode_and_evaluate_pointnet.py

Results

CIFAR-10

VGG like net parameter:1.34M, val acc:85.16%, storage:5273KB

VGG like net
compress rate parameters val acc storage
0.9 0.134M 71.77% 306KB
0.8 0.268M 84.23% 565KB
0.7 0.402M 86.18% 825KB
0.6 0.536M 86.98% 1085KB
0.5 0.67M 86.54% 1344KB

ResNet-50 parameter:0.762M, val acc:93.79%, storage:3366KB

ResNet-50
compress rate parameters val acc storage
0.9 0.076M 90.06% 410KB
0.8 0.152M 90.96% 557KB
0.7 0.229M 90.72% 703KB
0.6 0.305M 91.07% 851KB
0.5 0.381M 90.50% 996KB

Person segmentation

Refered repo: Person-Segmentation-Keras

Unet parameter:9.55M, val acc:98.46%, storage:37410KB

Unet
compress rate parameters val acc storage
0.95 0.478M 91.93% 936KB
0.9 0.955M 98.11% 1877KB
0.8 1.91M 98.45% 3741KB
0.7 2.87M 98.50% 5612KB
0.6 3.82M 98.51% 7482KB
0.5 4.78M 98.53% 9354KB

PointNet

Refered repo:PointNet-Keras

PointNet parameter:3.49M, val acc:88.23%, storage:13772KB

PointNet
compress rate parameters val acc storage
0.9 0.349M 83.80% 921KB
0.8 0.698M 87.21% 1597KB
0.7 1.05M 87.18% 2269KB
0.6 1.4M 88.11%% 2945KB
0.5 1.75M 88.35% 3607KB