/TransformerFromScratch

In this repository I implement a transformer model from scratch using this tutorial:

Primary LanguagePython

Introduction

In this repository I implement a transformer model from scratch using pytorch. This implementation is based on the following tutorial, blog post and paper:

A transformer is implemented from scratch using this tutorial:
https://www.youtube.com/watch?v=U0s0f995w14

It is also helpful to read this blog-post:
http://peterbloem.nl/blog/transformers

The original paper:
https://arxiv.org/abs/1706.03762

Prerequsites

You need to have CUDA installed as well as the appropriate version of Pytorch. This can be done following this guide:
https://pytorch.org/get-started/locally/