/transformers-stuff

Codes, scripts, and notebooks on various aspects of transformer models.

Primary LanguageJupyter Notebook

transformers-stuff

This repository contains codes, scripts, and notebooks on various aspects of transformer models. Implementation uses PyTorch.

  • GPT_Transformer.ipynb contains a detailed walk-through of the transformer implemenatation while explaining core concepts along the way.
  • GPT_Language_Model.ipynb builds and trains a language model from scratch using wikipedia data, included relevant pre-processing of the data.