FinABSA is a T5-Large model trained for Aspect-Based Sentiment Analysis specifically for financial domains. The model was trained on the SEntFiN 1.0 and with several data augmentation tricks. Unlike traditional sentiment analysis models which predict a single sentiment label for each sentence, FinABSA has been trained to disambiguate sentences containing multiple aspects. By replacing the target aspect with a [TGT] token the model predicts the sentiment concentrating to the aspect.
In this sector, we compare FinABSA againset FinBERT, the most widely used pretrained language model in the finance domain. As shown in the table below due to the structual limitations of the task itself FinBERT fails to correctly classify sentences with contradicting sentiments.
| SRC | Model | Prediction |
|---|---|---|
| Tesla stocks dropped 42% while Samsung rallied. | FinBERT | NEGATIVE |
| [TGT] stocks dropped 42% while Samsung rallied. | FinABSA | NEGATIVE |
| Tesla stocks dropped 42% while [TGT] rallied. | FinASBA | POSITIVE |
FinABSA is supported in two models, T5-Large and DeBERTa-v3-Base. T5 was trained using a prompt generation method similar to that of Aspect Sentiment Quad Prediction as Paraphrase Generation. DeBERTa was trained using a conventional sequence classification objective. We observe that the DeBERTa model achieves a slightly higher accuracy compared to the T5 model. All models are available at huggingface.
!git clone https://github.com/guijinSON/FinABSA.git
!pip install transformers
!pip install flairfrom FinABSA.FinABSA import ABSA
import pprint
absa = ABSA()
o = absa.run_absa(
input_str = "Economic headwinds also add to uncertainties. Major companies, including Alphabet, Apple, Microsoft, and Meta, have indicated a \
slowing pace of hiring for the rest of the year, suggesting mega-tech firms are bracing for a more uncertain economic outlook. \
Tougher macroeconomic conditions are likely to lead to cuts in advertising budgets, while the squeeze on household disposable income through \
inflation should weigh on discretionary consumer spending—both factors will hurt e-commerce and digital media companies. Meanwhile, reduced capital \
expenditure and elevated inventories will likely weigh on semiconductor and hardware companies.."
)
pprint.pprint(o)
# {'Alphabet': {'classification_output': 'NEGATIVE',
# 'logits': {'negative': 0.9970308542251587,
# 'neutral': 0.0018199979094788432,
# 'positive': 0.0011491376208141446}},
# 'Apple': {'classification_output': 'NEGATIVE',
# 'logits': {'negative': 0.9980792999267578,
# 'neutral': 0.0013628738233819604,
# 'positive': 0.000557745574042201}},
# 'Meta': {'classification_output': 'NEGATIVE',
# 'logits': {'negative': 0.9947644472122192,
# 'neutral': 0.004664959851652384,
# 'positive': 0.0005706017836928368}},
# 'Microsoft': {'classification_output': 'NEGATIVE',
# 'logits': {'negative': 0.9938719272613525,
# 'neutral': 0.005691188853234053,
# 'positive': 0.00043679968803189695}}}from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
model = AutoModelForSeq2SeqLM.from_pretrained("amphora/FinABSA")
tokenizer = AutoTokenizer.from_pretrained("amphora/FinABSA")input_str = "[TGT] stocks dropped 42% while Samsung rallied."
input = tokenizer(input_str, return_tensors='pt')
output = model.generate(**input, max_length=20)
print(tokenizer.decode(output[0]))
# The sentiment for [TGT] in the given sentence is NEGATIVE.The model shows lower performance as the input sentence gets longer, mostly because the dataset used to train the model consists of source sentences with short sequences. To apply the model on longer sequences try amphora/FinABSA-Longer.
If you use this work, please consider citing:
@article{son2023removing,
title={Removing non-stationary knowledge from pre-trained language models for entity-level sentiment classification in finance},
author={Son, Guijin and Lee, Hanwool and Kang, Nahyeon and Hahm, Moonjeong},
journal={arXiv preprint arXiv:2301.03136},
year={2023}
}
Feel free to reach me at spthsrbwls123@yonsei.ac.kr