/pytorch-sentiment-analysis

Tutorials on getting started with PyTorch and TorchText for sentiment analysis.

Primary LanguageJupyter NotebookMIT LicenseMIT

PyTorch Sentiment Analysis

This repo contains tutorials covering how to do sentiment analysis using PyTorch 0.4 and TorchText 0.3 using Python 3.6.

The first 2 tutorials will cover getting started with the de facto approach to sentiment analysis: recurrent neural networks (RNNs). The third notebook covers the FastText model and the final covers a convolutional neural network (CNN) model.

There are also 2 bonus "appendix" notebooks. The first covers loading your own datasets with TorchText, while the second contains a brief look at the pre-trained word embeddings provided by TorchText.

If you find any mistakes or disagree with any of the explanations, please do not hesitate to submit an issue. I welcome any feedback, positive or negative!

Getting Started

To install PyTorch, see installation instructions on the PyTorch website.

To install TorchText:

pip install torchtext

We'll also make use of spaCy to tokenize our data. To install spaCy, follow the instructions here making sure to install the English models with:

python -m spacy download en

Tutorials

  • 1 - Simple Sentiment Analysis

    This tutorial covers the workflow of a PyTorch with TorchText project. We'll learn how to: load data, create train/test/validation splits, build a vocabulary, create data iterators, define a model and implement the train/evaluate/test loop. The model will be simple and achieve poor performance, but this will be improved in the subsequent tutorials.

  • 2 - Upgraded Sentiment Analysis

    Now we have the basic workflow covered, this tutorial will focus on improving our results. We'll cover: loading and using pre-trained word embeddings, different optimizers, different RNN architectures, bi-directional RNNs, multi-layer (aka deep) RNNs and regularization.

  • 3 - Faster Sentiment Analysis

    After we've covered all the fancy upgrades to RNNs, we'll look at a different approach that does not use RNNs. More specifically, we'll implement the model from Bag of Tricks for Efficient Text Classification. This simple model achieves comparable performance as the Upgraded Sentiment Analysis, but trains much faster.

  • 4 - Convolutional Sentiment Analysis

    Next, we'll cover convolutional neural networks (CNNs) for sentiment analysis. This model will be an implementation of Convolutional Neural Networks for Sentence Classification.

  • 5 - Multi-class Sentiment Analysis

    Finally, we'll cover the case where we have more than 2 classes, as is common in NLP. We'll be using the CNN model from the previous notebook and a new dataset which has 6 classes.

Appendices