Implementation of the ideas proposed in Exploring Optimism and Pessimism in Twitter Using Deep Learning, (C. Caragea et al. 2018)
Analysis of the Optimism/Pessimism present in a tweet using Deep Learning methods.
The study focuses on the Twitter Optimism (OPT) dataset proposed in Finding Optimists and Pessimists on Twitter (Ruan et al. 2016). In this project we follow the methodology proposed in Exploring Optimism and Pessimism in Twitter Using Deep Learning, (C. Caragea et al. 2018).
Download pre-trained embeddings and data:
mkdir optimism && cd optimism
wget http://web.eecs.umich.edu/~mihalcea/downloads/optimism-twitter-data.zip data/
unzip data/optimism-twitter-data.zip data/
wget http://nlp.stanford.edu/data/glove.twitter.27B.zip embeddings/
unzip glove.twitter.27B.zip embeddings/
Clone project & prepare environment:
git clone https://github.com/StefanCobeli/Exploring-Optimism.git
conda env create -f environment.yml
conda activate nlp
Run minimal project (train a Deep Learning model on the OPT dataset):
python main.py -c config/initial_configs/OPT_small_RAM
- Trained models on OPT on Tweet Level on both 0 & 1/-1 settings (Section 3.1);
- Used BiLSTM, GRUStack & CNN as encoders.
- Used multiple types of static word embedding:
- Trained on TSA and tested on OPT (Section 3.2);
- First try on finetunig TSA trained model on OPT.
- Results (accuracy on OPT) are similar when using any type of encoding & embedding;
- Large training batch size (>2048) seem to stuck training performance in local minima;
- After 1 training epoch on TSA, accuracy on OPT is ~ 0.74;
- Freezing pre-trained embedding weights provide more stable accuracy on validation set (prevent overfitting on sentiment);
- TSA tweets can be trimmed from length ~100 to ~30, without loss in accuracy;
- Need to investigate how to perform knowledge transfer between TSA and OPT. Training only the final MLPs of the architecture (freezing the others, embedding & encoding) seems to work better.
Training Deep Learning architecture on Twitter Sentiment Analysis (TSA) dataset and validating results on Twitter Optimism (OPT) dataset.
