Building Transformers from scratch for regression and classification tasks. The modules include:
Multi-head Attention
Transformer Block(s)
Positional Encoding
Encoder / Decoder
An EncoderTransformer Architecture developed from scratch using pytorch's neural network module as the base class. The developed model used for sentiment analysis and time series prediction tasks.
Jupyter Notebook