/pytorch-progression

Worked examples of PyTorch from linear regression to transfer learning on NLP

Primary LanguageJupyter Notebook

pytorch-progression

Worked examples of PyTorch from linear regression to transformers, with a focus on NLP tasks.

Plan:

  • Linear regression with plain PyTorch
  • Linear regression with PyTorch Ignite
  • Logistic regression
  • Simple Neural Networks
  • Word2Vec
  • Deep Neural Network
  • Common tricks for training DNNs
    • Dropout
    • BatchNorm, LayerNorm
    • Early Stopping
  • Recurrent Neural Networks
  • RNNs + attention: LSTMs/GRUs
  • Stacking and BiDirectional
  • Seq2Seq
  • Transformers
  • Transfer learning with transformers