Neuron_view Asafaya pretrained model

[hinnaweali]

Hello,

We appreciate your assistance with this helpful visualization for Bert.
This issue occurs when I use the Asafaya pretrained model for the Arabic language, but not when I use the bert-base-multilingual-cased model.

Any suggestions!

best,

[jessevig]

Hi @hinnaweali , this seems to work, though please check the output. Please let me know if it works for you.

Note that you would need to run it with the exact order of imports shown (there are two versions of BertModel that I'm using here).

from transformers import BertModel, BertTokenizer
model = BertModel.from_pretrained('asafaya/bert-base-arabic', output_attentions=True)
model.save_pretrained('./model_neuron_view_test/')

from bertviz.transformers_neuron_view import BertModel
model = BertModel.from_pretrained('./model_neuron_view_test', output_attentions=True)
tokenizer = BertTokenizer.from_pretrained('asafaya/bert-base-arabic')

from bertviz.neuron_view import show
sentence_a = sentence_b = "هذا اختبار"
show(model, 'bert', tokenizer, sentence_a, sentence_b, display_mode='dark', layer=2, head=0)

[hinnaweali]

Hello Jesse,

I'd want to thank you for your helpful response.
Another issue arises, this time involving javascript visualization.
For example, using the show function with any head value yields no visual results. It is only possible to view a map with a layer value assigned. In addition, Also, the visualization map's drop-down list only allows me to choose a layer/head of 0.

best,
Ali

[jessevig]

Hmm, I see @hinnaweali . Could you paste in your code and I can try to run it just as you have it?

[hinnaweali]

It is the same code you sent. I only changed the sentences plus the layer and head values.

from transformers import BertModel, BertTokenizer
model = BertModel.from_pretrained('asafaya/bert-base-arabic', output_attentions=True)
model.save_pretrained('./model_neuron_view_test/')

from bertviz.bertviz.transformers_neuron_view import BertModel
model = BertModel.from_pretrained('./model_neuron_view_test', output_attentions=True)
tokenizer = BertTokenizer.from_pretrained('asafaya/bert-base-arabic')

from bertviz.bertviz.neuron_view import show
sentence_a = "طريقة إستعمال قطرة الانف للإطفال"
sentence_b = " اذا كانت قطر للتنظيف استعمل عند الحاجة قبل الرضعات و قبل النوم"
show(model, 'bert', tokenizer, sentence_a, sentence_b, display_mode='light', layer = 11, head = 6)

[jessevig]

Hi @hinnaweali this is what I get:

What browser are you running on? Also, are you using vanilla jupyter notebook?

[hinnaweali]

It seems that I should use Jupyter Lab instead! I ran your code in both Microsoft Edge and Google Chrome.
Thank you very much.

I used to run the script in both browsers using Bert Multilingual cased model and I did not get any visualization problem.

[jessevig]

Hmm, I've been able to run it on regular Jupyter using Chrome. There might be issues with Microsoft Edge though. Are you able to run the neuron view in this standard notebook without changing anything?: https://github.com/jessevig/bertviz/blob/master/notebooks/neuron_view_bert.ipynb Also, which version of Transformers are you using? I'm using transformers==4.9.1. Curious if you can get your model to load in the head or model views (should just be able to substitute your model name in the standard notebook, e.g.: https://github.com/jessevig/bertviz/blob/master/notebooks/head_view_bert.ipynb

…

On Sat, Jul 31, 2021 at 10:01 AM hinnaweali ***@***.***> wrote: It seems that I should use Jupyter Lab instead! I ran your code on both Microsoft Edge and Google Chrome. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#77 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKZXYVNB5M2KDVDVNYEQJZLT2QT5ZANCNFSM5BETYJ3Q> .

[hinnaweali]

Hello Jesse,

Please accept my apologies for the delayed response.
The issue has been resolved by upgrading the transformer version of the Jupyter notebook from 4.4.2 to 4.9.1, using both Google Chrome and Microsoft Edge.

Regards,

<style> </style>

Visualization Type	Browser	Transformer version	Status	Kernel
neuron view	Edge	4.9.1	done successfuly	Jupyter Notebook
head view	Edge	4.9.1	done successfuly	Jupyter Notebook
Model view	Edge	4.9.1	done successfuly	Jupyter Notebook
neuron view	Chrome	4.9.1	done successfuly	Jupyter Notebook
head view	Chrome	4.9.1	done successfuly	Jupyter Notebook
Model view	Chrome	4.9.1	done successfuly	Jupyter Notebook

[jessevig]

Great! Thanks for the update.