- One implementation of the paper Multi-label and Multi-target Sampling of Machine Annotation for Computational Stance Detection in EMNLP 2023
- This repo and the enriched annotation are only for research use. Please cite the papers if they are helpful.
-
The scripts were tested on following libraries and versions:
python==3.9
transformers==4.31
torch==1.13.1 -
To run the adversarial strategy of multi-target machine annotation, please install spacy and benepar for the parsing step:
https://spacy.io/models/en#en_core_web_trf
https://github.com/nikitakit/self-attentive-parser
- The original SemEval16 Tweet Stance Detection data are located at
./Stance_Data_JSON/
. - The machine annotation of the SemEval16 Tweet Stance Detection (Task A) is located at
./Machine_Annotation/
. - The stance label value 1 denotes 'Favor', 0 denotes 'Against', and 2 denotes 'None'.
- Run the
Model_as_Service.py
to start the LLM service and parsing model. - Run the
Machine_Annotation_main.py
for your experiments on machine annotation for stance detection data. - You can convert other stance detection corpora to json format.
- The default LLM as example is
vicuna-13b-v1.5
, you can change thecheckpoint_path
to use other models in theModel_as_Service.py
. Please adjust the prompt format according to the LLMs you use.
If the work is helpful, please cite our papers in your reports, slides, and papers.
@inproceedings{liu-etal-2023-multi,
title = "Multi-label and Multi-target Sampling of Machine Annotation for Computational Stance Detection",
author = "Liu, Zhengyuan and
Chieu, Hai Leong and
Chen, Nancy",
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2023",
month = dec,
year = "2023",
address = "Singapore",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.findings-emnlp.174",
doi = "10.18653/v1/2023.findings-emnlp.174",
pages = "2641--2649",
}
@inproceedings{liu-etal-2023-guiding,
title = "Guiding Computational Stance Detection with Expanded Stance Triangle Framework",
author = "Liu, Zhengyuan and
Yap, Yong Keong and
Chieu, Hai Leong and
Chen, Nancy",
booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = jul,
year = "2023",
address = "Toronto, Canada",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.acl-long.220",
doi = "10.18653/v1/2023.acl-long.220",
pages = "3987--4001",
}