Language Model (Rough work)
NOT WELL DOCUMENTED
Using Transformer to build Language Model. Still work in progress.
The maing goal is to implement something similar to GPT2. but for now just ussing the same archticture in the original Transformer model for language modeling.
The GP.ipynb , I just remove the encoder and re-write some part of the code from havard nlp code.
To understand the code checkout the havard nlp link above and also checkout the GP.ipynb. All other ,ipynb are not commented. But they all follow the same pattern as GP.ipynb and for better understanding checkout the medium post.