Transformers have revolutionised the AI space particularly NLP. It has improved areas such as;
- Machine translation.
- Next word prediction.
- Text classification and so much more.
In this repo, the transformer architecture is implemented; Starting from
- Positional encoding
- Scaled dot product
- Multihead attention
- Encoder
- Decoder
References