How to use the code for ADI?
theAverageArchit opened this issue · 1 comments
I am having trouble figuring out how to use the code for performing ADI. What are the settings that we need to set for performing ADI?
Hi @theAverageArchit ,
I found that in the ImageNet implementation in this repository, you can set --adi_scale to non-zero (e.g. --adi_scale=0.2). This option activates a pre-trained Resnet18 as a student and activates the ADI loss computation. In this case, the ResNet18 weights are fixed (there is no backpropagated gradient to the ResNet18).
However, it seems to me that this is different from what is described in the paper: in the paper, the student model is initialized from scratch and is modified by gradients.
Can anyone explain this?
After all, is there a full implementation of ADI in this repository? Wouldn't I need to use knowledge distillation from teacher to student in order to enable ADI, as described in the paper?