/EncoderTransformerArchitecture-FromScratch

An EncoderTransformer Architecture developed from scratch using pytorch's neural network module as the base class. The developed model used for sentiment analysis and time series prediction tasks.

Primary LanguageJupyter Notebook

EncoderTransformerArchitecture (BERT)

Building Transformers from scratch for regression and classification tasks. The modules include:

Multi-head Attention

Transformer Block(s)

Positional Encoding

Encoder / Decoder