This repository is the official PyTorch implementation of:
TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation (ICCV 2023).
This paper focuses on an under-explored yet important problem: Federated Class-Continual Learning (FCCL), where new classes are dynamically added in federated learning.
Existing FCCL works suffer from various limitations, such as requiring additional datasets or storing the private data from previous tasks. In response, we first demonstrate that non-IID data exacerbates catastrophic forgetting issue in FL. Then we propose a novel method called TARGET (federatTed clAss-continual leaRninG via Exemplar-free disTillation), which alleviates catastrophic forgetting in FCCL while preserving client data privacy. Our proposed method leverages the previously trained global model to transfer knowledge of old tasks to the current task at the model level. Moreover, a generator is trained to produce synthetic data to simulate the global distribution of data on each client at the data level. Compared to previous FCCL methods, TARGET does not require any additional datasets or storing real data from previous tasks, which makes it
ideal for data-sensitive scenarios.
We test the code on V100 GPU with pytorch:
torch==2.0.1
torchvision==0.15.2
Here, we provide a simple example for different methods.
For example, for cifar100-5tasks, please run the following commands to test the model performance with non-IID ($\beta=0.5$) data.
#!/bin/bash
# method= ["finetue", "lwf", "ewc", "icarl"]
CUDA_VISIBLE_DEVICES=0 python main.py --wandb=1 --group=5tasks_cifar100 --method=$method --tasks=5 --beta=0.5
CUDA_VISIBLE_DEVICES=0 python main.py --wandb=1 --group=5tasks_cifar100 --method=ours --tasks=5 --beta=0.5 --nums=8000 --kd=25 --exp_name=25_kd_8k_data
@misc{zhang2023addressing,
title={Addressing Catastrophic Forgetting in Federated Class-Continual Learning},
author={Jie Zhang and Chen Chen and Weiming Zhuang and Lingjuan Lv},
year={2023},
eprint={2303.06937},
archivePrefix={arXiv},
primaryClass={cs.LG}
}