This repository contains code for the implementation of RIFLE using Paddlepaddle.
[ICML'20] Xingjian Li*, Haoyi Xiong*, Haozhe An, Dejing Dou, and Cheng-Zhong Xu. RIFLE: Backpropagation in Depth for Deep Transfer Learning through Re-Initializing the Fully-connected LayEr. International Conference on Machine Learning (ICML’20), Vienna, Austria, 2020.
- Download transfer learning target datasets, like CUB_200_2011, Stanford Cars or others. Arrange the dataset in this way:
root/train/dog/xxy.jpg
root/train/dog/xxz.jpg
...
root/train/cat/nsdf3.jpg
root/train/cat/asd932_.jpg
...
root/test/dog/xxx.jpg
...
root/test/cat/123.jpg
...
- Download the pretrained models. We report the results of ResNet-50 below.
Modify global_data_path in as_data_reader/data_path.py to the path root where the dataset is.
Below are the commands to run baseline and RIFLE on Stanford Cars respectively,
python run.py --dataset Stanford_Cars --num_epoch 40 --batch_size 32 --fc_reinit 0
python run.py --dataset Stanford_Cars --num_epoch 40 --batch_size 32 --fc_reinit 1 --cyclic_num 2 Modify the argument appropriately to adjust the dataset folder name and the number of re-initializations of FC layer as desired.
We test cyclic_num = [2,3,4] and report the best results for RIFLE among the three. In this version of implementation, we avoid any re-initialization of FC layer when the training enters its last 5 epochs, so as to prevent the final model from being underfitted due to the lack of training budget after the last re-initialization.
| Dataset | l2 | RIFLE |
|---|---|---|
| CUB_200_2011 | 0.7487 | 0.7618 (cyclic_num=3) |
| Stanford-Cars | 0.8492 | 0.8610 (cyclic_num=4) |