Fork by Giuseppe Attardi from https://github.com/yoonkim/CNN_sentence.git. Differences with the original:
- multiclass classification
- model saving and loading for classifying files, e.g. tweets
- configurable parameters
- integrated program, without a separate preprocessing stage
- input in a single file, annotated in the format used in the SemEval Twitter Sentiment Analysis tasks.
The original code by Kim Yoon implements the technique described in the paper: Convolutional Neural Networks for Sentence Classification (EMNLP 2014). Please cite both this page and the original paper when using the program.
Code is written in Python (2.7) and requires Theano (0.7).
Using the pre-trained word2vec vectors will also require downloading the binary file from
https://code.google.com/p/word2vec/
usage: conv_net_tweets.py [-h] [-train] [-vectors VECTORS] [-filters FILTERS]
[-lower] [-dropout DROPOUT] [-epochs EPOCHS]
[-tagField TAGFIELD] [-textField TEXTFIELD]
model input
CNN sentence/tweet classifier.
positional arguments:
model model file (default mr)
input train/test file in SemEval twitter format
optional arguments:
-h, --help show this help message and exit
-train train model
-lower whether to lowercase text
-filters FILTERS n[,n]* (default 3,4,5)
-vectors VECTORS word2vec embeddings file (random values if missing)
-dropout DROPOUT dropout probability (default 0.5)
-epochs EPOCHS training iterations (default 25)
-tagField TAGFIELD label field in files (default 1)
-textField TEXTFIELD text field in files (default 2)
VECTORS is the word2vec binary file (e.g. GoogleNews-vectors-negative300.bin file),
clean, if present, text is lower-cased.
input contain sentences from the corpus in SemEval format, i.e. one sentence per line, tab separated values:
ID UID text label
Also a sentence classifier is present, that does cross validation on the training set for evaluating the performance of the classifier, like in the original code:
conv_net_sentence.py [-h] [-train] [-vectors VECTORS] [-filters FILTERS]
[-lower] [-dropout DROPOUT] [-epochs EPOCHS]
[-tagField TAGFIELD] [-textField TEXTFIELD]
model input
Example commands:
THEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float32 ./conv_net_sentence.py -vectors w2v_file mr.w2v mr.tsv
This will run the CNN classifier using the embeddings from file w2v_file and the sentences in the TSV file mr.tsv and saving the model to file mr.w2v.
GPU will result in a good 10x to 20x speed-up, so it is highly recommended.
To use the GPU, simply change device=cpu to device=gpu (or whichever gpu you are using).
For example:
THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python conv_net_sentence.py -nonstatic -word2vec
CPU output:
epoch: 1, training time: 219.72 secs, train perf: 81.79 %, val perf: 79.26 %
epoch: 2, training time: 219.55 secs, train perf: 82.64 %, val perf: 76.84 %
epoch: 3, training time: 219.54 secs, train perf: 92.06 %, val perf: 80.95 %
GPU output:
epoch: 1, training time: 16.49 secs, train perf: 81.80 %, val perf: 78.32 %
epoch: 2, training time: 16.12 secs, train perf: 82.53 %, val perf: 76.74 %
epoch: 3, training time: 16.16 secs, train perf: 91.87 %, val perf: 81.37 %
Denny Britz has an implementation of the model in TensorFlow:
https://github.com/dennybritz/cnn-text-classification-tf
He also wrote a nice tutorial on it, as well as a general tutorial on CNNs for NLP.
Ye Zhang has written a very nice paper doing an extensive analysis of model variants (e.g. filter widths, k-max pooling, word2vec vs Glove, etc.) and their effect on performance.
Experiments using the data from SemEval-15, using option --filters 7,7,7, achieves an accuracy of 67.28%, better than the top official score (64.84%), as reported in: Sara Rosenthal et al. SemEval-2015 Task 10: Sentiment Analysis in Twitter.