ohsuz/dekorde

트랜스포머 밑바닥부터 구현 - 수업 & Sesame's Pirates

dekorde

---

✏️ Pre-requisites

• torch.einsum
  Image Source : https://baekyeongmin.github.io/dev/einsum/

  • https://pytorch.org/docs/stable/generated/torch.einsum.html
  • https://baekyeongmin.github.io/dev/einsum/
  • https://ita9naiwa.github.io/numeric calculation/2018/11/10/Einsum.html

• torch.tensor.expand
  

  • https://pytorch.org/docs/stable/generated/torch.Tensor.expand.html
  • https://seducinghyeok.tistory.com/9

• torch.tril

  • https://pytorch.org/docs/stable/generated/torch.tril.html

• torch.tensor.masked_fill

  • https://pytorch.org/docs/stable/generated/torch.Tensor.masked_fill_.html#torch.Tensor.masked_fill_

---

🔗 Useful Links

• https://www.youtube.com/watch?v=U0s0f995w14
• https://github.com/hyunwoongko/transformer
• https://github.com/tunz/transformer-pytorch
• https://github.com/paul-hyun/transformer-evolution
• https://github.com/jadore801120/attention-is-all-you-need-pytorch