TAS-BERT

Code and data for our paper Target-Aspect-Sentiment Joint Detection for Aspect-Based Sentiment Analysis" (AAAI 2020)

Our code is based on Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence (NAACL 2019)

Requirements

pytorch: 1.0.1
python: 3.6.8
tensorflow: 1.13.1 (only for creating BERT-pytorch-model)
pytorch-crf: 0.7.2
numpy: 1.16.4
nltk: 3.4.4
sklearn: 0.21.2

Data Preprocessing

Download uncased BERT-Based model, unpack, place the folder in the root directory and run convert_tf_checkpoint_to_pytorch.py to create BERT-pytorch-model.

run following commands to get preprocessed data.

cd data
python data_preprocessing_for_TAS.py --dataset semeval2015
python data_preprocessing_for_TAS.py --dataset semeval2016
cd ../

The preprocessed data is in folders semeval2015/three_joint/BIO，semeval2015/three_joint/TO，semeval2016/three_joint/BIO and semeval2016/three_joint/TO. BIO and TO are the two tagging schemes mentioned in our paper.

The preprocessed data structure is as follows:

Key	Description
sentence_id	Id of the sentence.
yes_no	whether the sentence has corresponding sentiment in the corresponding aspect. The corresponding sentiment and aspect are given in aspect_sentiment.
aspect_sentiment	<aspect, sentiment> pair of this line, such as "food quality positive".
sentence	Content of the sentence.
ner_tags	label sequence for targets that have corresponding sentiment in the corresponding aspect. The corresponding sentiment and aspect are given in aspect_sentiment.

Code Structure

TAS_BERT_joint.py: Program Runner.
modeling.py: Program Models.
optimization.py: Optimization for model.
processor.py: Data Processor.
tokenization.py: Tokenization, including three unknown-word-solutions.
evaluation_for_TSD_ASD_TASD.py: evaluation for ASD, TASD and TSD tasks.
evaluation_for_AD_TD_TAD/: The official evaluation tool for AD, TD and TAD tasks.
TAS_BERT_separate.py and evaluation_for_loss_separate.py: Separate detection for Ablation Study.

Training & Testing

If you want to train and test a joint detection model, you can use the following command:

CUDA_VISIBLE_DEVICES=0 python TAS_BERT_joint.py \
--data_dir data/semeval2016/three_joint/BIO/ \
--output_dir results/semeval2016/three_joint/BIO/my_result \
--vocab_file uncased_L-12_H-768_A-12/vocab.txt \
--bert_config_file uncased_L-12_H-768_A-12/bert_config.json \
--init_checkpoint uncased_L-12_H-768_A-12/pytorch_model.bin \
--tokenize_method word_split \
--use_crf \
--eval_test \
--do_lower_case \
--max_seq_length 128 \
--train_batch_size 24 \
--eval_batch_size 8 \
--learning_rate 2e-5 \
--num_train_epochs 30.0

The test results for each epoch will be stored in test_ep_*.txt in the output folder.

Evaluation

If you want to evaluate the test result for each epoch, you can use the following commands:

Note: We chose the epoch which performed best on the TASD task, and evaluate the result on all the subtasks.

If you want to evaluate on the TASD task, ASD task, TSD task ignoring implicit targets, and TSD task considering implicit targets, you can use the following command:
```
python evaluation_for_TSD_ASD_TASD.py \
--output_dir results/semeval2016/three_joint/BIO/my_result \
--num_epochs 30 \
--tag_schema BIO
```

⭐The tag_schema bust be consistent with the contents in the output_dir, otherwise you will get error results.

"All tuples" correspond to "C1" in Table 3 of our paper.

"Only NULL tuples" correspond to "C2" in Table 3 of our paper.

"NO and pure O tag sequence" correspond to "C3" in Table 3 of our paper.

As for the TD, AD and TAD tasks, we use the evaluation tool provided by the SemEval2015 competition. The tool requires a Java environment.

First, we should convert our test results into XML file format. You can use the following command:

cd evaluation_for_AD_TD_TAD
python change_pre_to_xml.py \
--gold_path ../data/semeval2016/three_joint/BIO/test_TAS.tsv \
--pre_path ../results/semeval2016/three_joint/BIO/my_result/test_ep_23.txt \
--gold_xml_file ABSA16_Restaurants_Test.xml \
--pre_xml_file pred_file_2016.xml \
--tag_schema BIO

Note: the "test_ep_*.txt" is the best epoch on the TASD task.

We will get a predication file in XML format: pred_file_2016.xml.

If you want to evaluate on the AD task:

java -cp ./A.jar absa15.Do Eval ./pred_file_2016.xml ./ABSA16_Restaurants_Test.xml 1 0

If you want to evaluate on the TD task:

java -cp ./A.jar absa15.Do Eval ./pred_file_2016.xml ./ABSA16_Restaurants_Test.xml 2 0

If you want to evaluate on the TAD task:

java -cp ./A.jar absa15.Do Eval ./pred_file_2016.xml ./ABSA16_Restaurants_Test.xml 3 0

Ablation Study

If you want to try the separate models, please use the following commands:

CUDA_VISIBLE_DEVICES=0 python TAS_BERT_separate.py \
--data_dir data/semeval2016/three_joint/BIO/ \
--output_dir results/semeval2016/three_joint/BIO/my_result_AS \
--vocab_file uncased_L-12_H-768_A-12/vocab.txt \
--bert_config_file uncased_L-12_H-768_A-12/bert_config.json \
--init_checkpoint uncased_L-12_H-768_A-12/pytorch_model.bin \
--tokenize_method word_split \
--use_crf \
--subtask AS \
--eval_test \
--do_lower_case \
--max_seq_length 128 \
--train_batch_size 24 \
--eval_batch_size 8 \
--learning_rate 2e-5 \
--num_train_epochs 30.0

CUDA_VISIBLE_DEVICES=0 python TAS_BERT_separate.py \
--data_dir data/semeval2016/three_joint/BIO/ \
--output_dir results/semeval2016/three_joint/BIO/my_result_T \
--vocab_file uncased_L-12_H-768_A-12/vocab.txt \
--bert_config_file uncased_L-12_H-768_A-12/bert_config.json \
--init_checkpoint uncased_L-12_H-768_A-12/pytorch_model.bin \
--tokenize_method word_split \
--use_crf \
--subtask T \
--eval_test \
--do_lower_case \
--max_seq_length 128 \
--train_batch_size 24 \
--eval_batch_size 8 \
--learning_rate 2e-5 \
--num_train_epochs 30.0

python evaluation_for_loss_separate.py \
--output_dir_AS results/semeval2016/three_joint/BIO/my_result_AS \
--output_dir_T results/semeval2016/three_joint/BIO/my_result_T \
--num_epochs 30 \
--tag_schema BIO