Error during attention intervention

[LopezGG]

Hi

I tried to replicate your code attention_intervention_winobias.py. I see the following error in attention_intervention_model.py

ModuleAttributeError: 'Attention' object has no attribute 'output_attentions

I am trying to debug. Should you know of a quick fix do let me know!

My huggingFace transformers version is '3.0.2'
Python version is 3.8
pytorch version is '1.6.0'
the full stack trace is

<ipython-input-90-e3cba5dcfce7> in intervention_hook(module, input, outputs, attn_override, attn_override_mask)
      1 outputs = []
      2 def intervention_hook(module, input, outputs, attn_override, attn_override_mask):
----> 3     attention_override_module = AttentionOverride(
      4         module, attn_override, attn_override_mask)
      5     outputs[:] = attention_override_module(*input)

/data/home/notebooks/Interpretability/CausalMediationAnalysis/attention_intervention_model.py in __init__(self, attention, attn_override, attn_override_mask)
     23         super(AttentionOverride, self).__init__()
     24         # Copy values from attention
---> 25         self.output_attentions = attention.output_attentions
     26         self.register_buffer("bias", attention._buffers["bias"])
     27         self.n_head = attention.n_head

~/anaconda3/envs/interpret_k1/lib/python3.8/site-packages/torch/nn/modules/module.py in __getattr__(self, name)
    769             if name in modules:
    770                 return modules[name]
--> 771         raise ModuleAttributeError("'{}' object has no attribute '{}'".format(
    772             type(self).__name__, name))
    773 

ModuleAttributeError: 'Attention' object has no attribute 'output_attentions'

[boknilev]

@jessevig could you take a look?

[jessevig]

Thank you @LopezGG for reporting this, and sorry for your troubles. It seems that v3 of Transformers is incompatible with the current codebase. Not sure if this will work for you, but if you downgrade to transformers 2.*, this should resolve the issue. I've updated requirements.txt to reflect this, but ideally we will put in a longer term fix at some point.

[LopezGG]

Thanks @jessevig and @boknilev . I will start with v2 to see how it works