sujitpal/deeplearning-ai-examples

nlp-deeplearning-ai-examples

---

This repository started as a place to store Colab notebooks for a Coursera NLP Specialization series that I was auditing. Because I am auditing, I don't have access to the exact assignments or datasets, so I improvised a bit, using the opportunity to try out appropriate illustrative toy examples. I also wanted to teach myself Pytorch, so all examples use Pytorch.

• Natural Language Processing with Classification and Vector Spaces.
  • Logistic Regression.
  • Sentiment Analysis.
  • Word Translation with Embeddings.
• Natural Language Processing with Probabilistic Models.
• Natural Language Processing with Sequence Models.
  • GPT2 based autocomplete and fine tuning email autocomplete.
  • BioNLP Named Entity Recognizer.
  • Siamese Network to detect Duplicate Questions.
• Natural Language Processing with Attention Models.
  • Neural Machine Translation using seq2seq + attention.
  • Transformer seq2seq for summarization.
  • Summarization with Pegasus (Inference only).
  • Question Answering with T5 (Inference and fine-tuning).
  • Multilabel Document classification with Longformer (fine-tuning).

---

Before I did that, though, I realized that I needed to learn Pytorch basics first, so I picked up an elementary Pytorch book (don't remember which unfortunately, it could be the Practical Deep Learning with Pytorch course on Udemy, and tried out some of the exercises with toy datasets I found here and there.

• Introduction to Pytorch notebook.
• Pytorch Building Blocks.
• Credit Card Repayment Classifier - FCN.
• CIFAR-10 Image Classifier - CNN.
• Time Series Prediction - RNN.

---

I also wanted to get familiar with Pytorch-Lightning, so I rebuilt the last 3 from the list above to use Pytorch Lightning.

• Credit Card Repayment Classifier - FCN + Lightning.
• CIFAR-10 Image Classifier - CNN + Lightning.
• Time Series Prediction - RNN + Lightning.

---

Here are some experiments with Longformer as part of research for an internal talk I did on Transformer Mods for Document Length Inputs (and also for the #nlp-embeddings group on the TWIML Slack Channel).

• Pretrained Longformer and Document Similarity.
• Fine-tuning Longformer for Sentiment Analysis.

---

Random notebooks, usually to back up blog posts or satisfy my own curiosity.

• Language Model or Knowledge Base? (Petroni, 2019).
  • Generating masked words with pre-trained BERT Language Model.
• Word Sense Disambiguation using BERT as a Language Model - blog post (Salmon Run, 2020).
  • WSD Using Raw BERT Embeddings.
  • WSD Using BERT Masked Language Model.