Attention Maps Visualization

[me-edu]

Hi. Thanks for the nice work!

I have a question about visualizing the attention maps. Do you use "visualize_attention.py" script? In that script, it seems that each attention head is visualized seperately. Do we need to average them? Is any thresholding used? I would be grateful if you provide the script used for producing the attention maps in the paper. Thanks

[maryam089]

Thank you so much for your interest in our work. Yes, we used "visualize_attention.py" script and we visualized attention maps separately from each head and selected the best result for our model. We did not average them. The threshold is empirically selected to be 0.8.