This repository enables the reproduction of the experiments described in the CVPR 2020 paper:
ReSprop: Reuse Sparsified Backpropagation
Negar Goli, Tor M. Aamodt
Link to our ReSprop paper in CVPR2020 Oral
-
Python 3.7.5
-
torch 1.1.0
-
torchvision
-
cuda 10
-
gcc 7.5.0
The code uses the custom C++ and cuda extensions of pytorch. CUSTOM C++ Pytorch
The c++ code is availabe in backward folder.
Build the costum c++ backward functions in the backward folder:
python setup.py install
Download ImageNet dataset and use uncropped data
The algorithm can be run on up to 8 GPUs
-a : Architecture --iterations: Number of iterations --sparsity: targeted reuse_sparsity --warmup: 1 means with warmup phase, 0 without
Example: ImageNet on resnet18 with 80% reuse sparsity and with warmup (100-80=20%).
python main_ImageNet.py -a resnet18 --sparsity 20 --warmup 1 PATH_TO_THE_IMAGENET_DATASET
ReSprop_conv.py is a costum convolution python kernel which includes the main part of ReSprop algorithm.
Please note that this code is showing the functionality of ReSprop algorithm and to gain the speedup a hardware accelerator is required.