seq-to-mind/Stance_MA

One implementation of the paper "Multi-label and Multi-target Sampling of Machine Annotation for Computational Stance Detection" (EMNLP 2023).

Introduction

• One implementation of the paper Multi-label and Multi-target Sampling of Machine Annotation for Computational Stance Detection in EMNLP 2023
  
• This repo and the enriched annotation are only for research use. Please cite the papers if they are helpful.
  

Package Requirements

• The scripts were tested on following libraries and versions:
  python==3.9
  transformers==4.31
  torch==1.13.1
  

• To run the adversarial strategy of multi-target machine annotation, please install spacy and benepar for the parsing step:
  https://spacy.io/models/en#en_core_web_trf
  https://github.com/nikitakit/self-attentive-parser
  

Data Format

• The original SemEval16 Tweet Stance Detection data are located at ./Stance_Data_JSON/.
  
• The machine annotation of the SemEval16 Tweet Stance Detection (Task A) is located at ./Machine_Annotation/.
  
• The stance label value 1 denotes 'Favor', 0 denotes 'Against', and 2 denotes 'None'.
  

Running Machine Annotation

• Run the Model_as_Service.py to start the LLM service and parsing model.
  
• Run the Machine_Annotation_main.py for your experiments on machine annotation for stance detection data.
  
• You can convert other stance detection corpora to json format.
  
• The default LLM as example is vicuna-13b-v1.5, you can change the checkpoint_path to use other models in the Model_as_Service.py. Please adjust the prompt format according to the LLMs you use.
  

Citation

If the work is helpful, please cite our papers in your reports, slides, and papers.

@inproceedings{liu-etal-2023-multi,
    title = "Multi-label and Multi-target Sampling of Machine Annotation for Computational Stance Detection",
    author = "Liu, Zhengyuan  and
      Chieu, Hai Leong  and
      Chen, Nancy",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2023",
    month = dec,
    year = "2023",
    address = "Singapore",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.findings-emnlp.174",
    doi = "10.18653/v1/2023.findings-emnlp.174",
    pages = "2641--2649",
}

@inproceedings{liu-etal-2023-guiding,
    title = "Guiding Computational Stance Detection with Expanded Stance Triangle Framework",
    author = "Liu, Zhengyuan  and
      Yap, Yong Keong  and
      Chieu, Hai Leong  and
      Chen, Nancy",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.220",
    doi = "10.18653/v1/2023.acl-long.220",
    pages = "3987--4001",
}