Attention visualization
Closed this issue · 5 comments
We need some attention visualization to show how attention gets better with our proposed method.
- Write a function for visualizing attention on a given
- Figure out which technique we should use for attention maps
- Sample a few random images and plot attention to them.
- @Magiccircuit will share a paper that he already considered for this task
Some motivation from here https://arxiv.org/pdf/2012.09838.pdf
check out this paper Generic attentionmodel explainability for interpreting bi-modal and encoderdecoder transformers., its for vision-language visualization, one thing to note is that the released code is for openai clip code, however we train clip model using openclip model, they are different, so you should try openclip model implementation
and we want something like this, this is from paper https://arxiv.org/abs/2302.12066
and we want something like this, this is from paper https://arxiv.org/abs/2302.12066
This one is from Google Research. They don't release codes :(
I mean we want some visualization like figures in this google paper, these visualization figures are obtained using techniques in https://arxiv.org/abs/2103.15679 this one, and this one release everything you need to visualize attention map given a word
I mean we want some visualization like figures in this google paper, these visualization figures are obtained using techniques in https://arxiv.org/abs/2103.15679 this one, and this one release everything you need to visualize attention map given a word
yep, right. thanks!