This repository contains codes, scripts, and notebooks on various aspects of transformer models. Implementation uses PyTorch.
GPT_Transformer.ipynb
contains a detailed walk-through of the transformer implemenatation while explaining core concepts along the way.GPT_Language_Model.ipynb
builds and trains a language model from scratch usingwikipedia
data, included relevant pre-processing of the data.