LUCIR in memoryless mode
HabibSlim opened this issue · 3 comments
In the code for LUCIR, we can read the following comment:
LUCIR is expected to be used with exemplars. If needed to be used without exemplars, overwrite here the _get_optimizer function with the one in LwF and update the criterion
How should the criterion function be updated to allow for a memoryless scenario? It is my understanding that the criterion function should already work without any exemplars in accordance to the original paper, but maybe I'm missing a detail.
(besides the fact that the margin ranking loss is unappliable of course)
We didn't test out the usage of LUCIR without exemplars, since the original paper makes use of them (I think they use at least 1 sample per class in all experiments, including the one measuring the effect of the memory size). In general, when a method would be used without exemplars, it would obtain better results if the access to previous heads is restricted during training of new tasks (specially if weight decay is being used). However, in LUCIR's original code, the less-forget loss constraint is already applied without modifying other heads already (as seen in the _get_optimizer() function).
Regarding the criterion, I agree that the margin ranking loss has to be removed (by using argument --remove-margin-ranking) and the current criterion should work already as it is provided. However, I would have to run a couple of experiments to make sure it doesn't break somewhere else.
Therefore, I believe that the comment in src/approach/lucir.py should be replaced by smth like:
LUCIR is expected to be used with exemplars. If needed to be used without exemplars, make sure to use the `--remove-margin-ranking` argument.
Thank you very much for the useful clarifications!
I'm going to proceed as you suggested and explicitly disable the margin ranking loss when running the code without exemplars.