The application of attention mechanism in the field of computer vision is mainly used to capture the respective field on the image, while the application in the field of natural language processing is mainly used to locate the key token. This project collects several attention mechanisms with concise code, which are very convenient to call and can be easily ported to your own network to improve performance.
There are a total of 8 attention mechanisms, among which fcanet.py and layer.py are the codes of the same method, and the rest of the attention mechanisms have only one .py file.
Attached below are links to papers or open source projects for these attention mechanisms:
SPA-Net:https://ieeexplore.ieee.org/document/9102906
https://github.com/Jongchan/attention-module/blob/master/MODELS/cbam.py
scSE:https://arxiv.org/abs/1803.02579
A2-Nets:https://papers.nips.cc/paper/7318-a2-nets-double-attention-networks.pdf