Implementing with my own model
LanariLorenzo opened this issue · 4 comments
Hi thank you very much for this tool it is amazing!
I would like to implement it in my own ViT model tough it has been heavily modified. I tried loading the state_dict of my model into your base ViT_LRP one but of course i got some errors due the differences.
Do you advise writing a specific load_state_dict or implementing relprop inside my blocks?
Thank you
Hi @LanariLorenzo, thanks for your interest in our work!
I think the best option would be to implement your own relprop as I'm not sure if your modifications are aligned with our implementation.
I would also suggest that you try our second method which uses only gradients, as it may be easier to implement.
Best,
Hila.
@hila-chefer Congratulations on great work! What's the main difference between this repo's work and here? And which performance is better when used into visualizing image transformer-based classification model?
Hi @XhqGlorry11, thanks for your interest!
Our second method eliminates the use of LRP for easier implementation and to reduce runtime.
We show on our second paper that both methods are very successful for self-attention networks (such as ViT). Using LRP can be beneficial in some cases, but overall both should be similar in most cases.
Best,
Hila.
@XhqGlorry11 closing this issue due to low activity, please reopen if necessary