Channel Pruning for Accelerating Very Deep Neural Networks

By Yihui He (Xi'an Jiaotong University), Xiangyu Zhang and Jian Sun (Megvii)
ICCV 2017

In this repository, we illustrate channel pruning VGG-16 4X with our 3C method. After finetuning, the Top-5 accuracy is 89.9% (suffers no performance degradation). Our 3C method is a combination of: (mentioned in 4.1.2)

spatial decomposition (Speeding up Convolutional Neural Networks with Low Rank Expansions)
channel decomposition (Accelerating Very Deep Convolutional Networks for Classification and Detection)
channel pruning


Structured simplification methods	Channel pruning (d)

Citation

If you find the code useful in your research, please consider citing:

@article{he2017channel,
  title={Channel Pruning for Accelerating Very Deep Neural Networks},
  author={He, Yihui and Zhang, Xiangyu and Sun, Jian},
  journal={arXiv preprint arXiv:1707.06168},
  year={2017}
}

Requirements
Installation
Channel Pruning and finetuning
Pruned models for download
FAQ

requirements

Python3 packages you might not have: scipy, sklearn, easydict, use sudo pip3 install to install.
For finetuning with 128 batch size, 4 GPUs (~11G of memory)

Installation (sufficient for the demo)

Clone the repository

# Make sure to clone with --recursive
git clone --recursive https://github.com/yihui-he/channel-pruning.git

Build my Caffe fork (which support bicubic interpolation and resizing image shorter side to 256 then crop to 224x224)

cd caffe

# If you're experienced with Caffe and have all of the requirements installed, then simply do:
make all -j8 && make pycaffe
# Or follow the Caffe installation instructions here:
# http://caffe.berkeleyvision.org/installation.html

# you might need to add pycaffe to PYTHONPATH, if you've already had a caffe before

Download ImageNet classification dataset http://www.image-net.org/download-images
Specify imagenet source path in temp/vgg.prototxt (line 12 and 36)

Channel Pruning

For fast testing, you can directly download pruned model. See next section

Download the original VGG-16 model http://www.robots.ox.ac.uk/~vgg/software/very_deep/caffe/VGG_ILSVRC_16_layers.caffemodel
move it to temp/vgg.caffemodel (or create a softlink instead)

Start Channel Pruning

python3 train.py -action c3 -caffe [GPU0]
# or log it with ./run.sh python3 train.py -action c3 -caffe [GPU0]
# replace [GPU0] with actual GPU device like 0,1 or 2

Combine some factorized layers for further compression, and calculate the acceleration ratio
```
./combine.sh | xargs ./calflop.sh
```

Finetuning

./finetune.sh [GPU0,GPU1,GPU2,GPU3]
# replace [GPU0,GPU1,GPU2,GPU3] with actual GPU device like 0,1,2,3

Testing Though testing is done while finetuning, you can test anytime with:

caffe test -model path/to/prototxt -weights path/to/caffemodel -iterations 5000 -gpu [GPU0]
# replace [GPU0] with actual GPU device like 0,1 or 2

Pruned models (for download)

For fast testing, you can directly download pruned model from release: https://github.com/yihui-he/channel-pruning/releases/download/VGG-16_3C4x/channel_pruning_VGG-16_3C4x.zip
Test with:

caffe test -model channel_pruning_VGG-16_3C4x.prototxt -weights channel_pruning_VGG-16_3C4x.caffemodel -iterations 5000 -gpu [GPU0]
# replace [GPU0] with actual GPU device like 0,1 or 2

FAQ

You can find answers of some commonly asked questions in our Github wiki, or just email me: yihuihe@foxmail.com

flygyyy/channel-pruning