/masked-language-modeling

Transformers Pre-Training with MLM objective — implemented encoder-only model and trained from scratch on Wikipedia dataset.

Primary LanguageJupyter Notebook

Stargazers