Negation bug?
Closed this issue · 2 comments
While evaluating for which model I should use for sentiment analysis, I found out that only 'fastformer' and 'tiny-fastformer' is free from this negation bug(not sure the correct term since I'm not from this domain). Probably because fastformer arch is better or they being trained on different dataset or different preprocessing(tokenization). Might be useful for error analysis. Feel free to close the issue if this is not a bug.
reproducible code:
https://colab.research.google.com/drive/1zXc1K1wUHNjVmYdOZZ3xFyuzrZRt5Sb7?usp=sharing
Thanks
actually fastformer also got bug, my previous evaluation code did not expect for neutral polarity
All these models pretrained on formal and social media structures, and context between formal and social media is very different.
And the models finetuned on actual human text data, example at https://malaya.readthedocs.io/en/latest/load-sentiment.html, the context is very different with hand crafted polarity text. I cannot give an actual explanation why it is sucks, but it can be,
- too much different context between formal and social media structures.
- heavily finetuned on social media texts.