Can you expand the lrp.py to cover BigBird?
ngoel17 opened this issue · 1 comments
ngoel17 commented
Most of the NLP tasks use BigBird due to long text support. Can you please extend the class to support BigBird, by adding rules for torch.nn.modules.sparse.Embedding etc?
hila-chefer commented
Hi @ngoel17, thank you for your interest!
We currently do not have plans to extend our implementation to BigBird, however, I'd suggest using the method from our second paper which does not require LRP (only gradient propagation).
Best,
Hila.