The official code of CPR: Classifier-Projection Regularization for Continual Learning (ICLR 2021) [arxiv]
$ pip install -r requirements.txt
$ mkdir weights data
- Download datasets (CIFAR[1], Omniglot[2] and CUB200[3]) from [this google drive link]
- Locate downloaded datasets to './data' directory
./data
/Permuted_Omniglot_task50.pt
/binary_split_cub200_new
/binary_split_cifar100
/binary_cifar10
/binary_omniglot
$ ./train_cifar.sh
$ ./train_omniglot.sh
$ ./train_cub200.sh
-
Check './result_analysis_code/'. There are example ipython files to anayze the experimental results of [EWC, MAS, SI, Rwalk, AGS-CL] with or without CPR in CIFAR100. Note that the analysis results are for experiments conducted on only single seed.
-
You can easily transform and use these files to analyze other results!
: The implementation for CPR is quite simple. As shown in Equation (3) of the paper, you can implement CPR by maximizing an entropy of a model's softmax output (in other words, minimizing KL divergence between the model's softmax output and uniform distribution). Note that a lambda (the hyperparameter for entropy maximization) should be selected carefully. As an example, check Line 222 at './approaches/ewc_cpr.py'.
@inproceedings{
cha2021cpr,
title={{\{}CPR{\}}: Classifier-Projection Regularization for Continual Learning},
author={Sungmin Cha and Hsiang Hsu and Taebaek Hwang and Flavio Calmon and Taesup Moon},
booktitle={International Conference on Learning Representations},
year={2021},
url={https://openreview.net/forum?id=F2v4aqEL6ze}
}
[1] Krizhevsky, Alex, and Geoffrey Hinton. "Learning multiple layers of features from tiny images." (2009): 7.
[2] Lake, Brenden M., Ruslan Salakhutdinov, and Joshua B. Tenenbaum. "Human-level concept learning through probabilistic program induction." Science 350.6266 (2015): 1332-1338.
[3] Welinder, Peter, et al. "Caltech-UCSD birds 200." (2010).