Attention_Based_Networks
Author : Shantam Bajpai
Description
With the Computer Vision community fast realizing the benefits of the transformer architecture to tackle computer vision problems this repository will be an assortment of implementations of various visual attention based networks starting from the famous transformer architecture from the paper " Attention is all you need" which was developed primarily for machine translation tasks but this architecture has formed the basis for the vision community to adopt the transformer architecture.
Research paper references
- Attention is all you need (Vanilla Transformer): https://arxiv.org/abs/1706.03762
- Vision Transformer: https://arxiv.org/pdf/2010.11929.pdf
- Data Efficient Image Transformer: https://arxiv.org/abs/2012.12877