Dootmaan/MT-UNet

包括External Attention原作者在内，似乎都没用共享两个记忆单元MK和MV

Closed this issue · 2 comments

fengyuchao97 commented

我能理解这个注意力模块的设计初衷是有意义的，但我看其源代码，以及作者您的代码里，仅对输入x做处理，并没有文章中提及的共享两个记忆单元MK和MV，不知道您是否注意到此问题，期待您的解答。

Dootmaan commented

hi @fyc1997 thank you for your question. mk and mv are in model/MTUNet.py line 116 & 117. They are two learnable matrices, and are implemented by two linear layers. This implementation is also adopted by the authors of External Attention.

  • 👍1
Dootmaan commented

This issue is closed since no further activity has happened for a while.