s-nlp/annotated-transformer

http://nlp.seas.harvard.edu/2018/04/03/attention.html

The Annotated Transformer

Code for The Annotated Transformer blog post:

http://nlp.seas.harvard.edu/2018/04/03/attention.html

---

We, SkoltechNLP group, revisited this notebook in April 2021 and made some adjusments to make it runnable like "Run All Cells" with updated required packages.

Aslo, we add some clarifing images from the amazing resources about Transformers that emerged during these years:

• Seq2Seq and Attention by Lena Voita
• The Illustrated Transformer

We believe that will add some additional understanding of the main idea and code. Enjoy your Transformer!

• Daryna Dementieva (@dementyeva_ds in any social network or daryna.dementieva@skoltech.ru)
• Anton Razzhigaev (anton.razzhigaev@skoltech.ru)
• Alexander Panchenko (a.pnachenko@skoltech.ru)