code for our NAACL 2019 paper "BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis".
We found that BERT domain post-training (e.g, 1 day of training) is an economic way to boost the performance of BERT, because it is much harder (e.g., 10 days of training) to learn a general knowledge shared across domains and, meanwhile, loosing the long-tailed domain-specific knowledge.
Code base on huggingface transformers
is under transformers
, with more cross-domain models.
Preprocessing ABSA xmls organized into a separate rep.
Want to have post-trained models for other domains in reviews ? checkout a cross-domain review BERT or download from HERE.
A conversational dataset of RRC can be found here.
If you only care about ASC, a more formal code base can be found in a similar rep focusing on ASC.
**feedbacks are welcomed for missing instructions **
We focus on 3 review-based tasks: review reading comprehension (RRC), aspect extraction (AE) and aspect sentiment classification (ASC).
RRC: given a question ("how is the retina display ?") and a review ("The retina display is great.") find an answer span ("great") from that review;
AE: given a review sentence ("The retina display is great."), find aspects("retina display");
ASC: given an aspect ("retina display") and a review sentence ("The retina display is great."), detect the polarity of that aspect (positive).
E2E-ABSA: the combination of the above two tasks as a sequence labeling task.
Now the code base is splited into two versions: transformers/
(instructions) and pytorch-pretrained-bert/
(instructions).
Please check corresponding instructions for details.
If you find this work useful, please cite as following.
@inproceedings{xu_bert2019,
title = "BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis",
author = "Xu, Hu and Liu, Bing and Shu, Lei and Yu, Philip S.",
booktitle = "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics",
month = "jun",
year = "2019",
}