czy1999/ARI-QA

ARI (Abstract Reasoning Induction) is an innovative framework designed to enhance the temporal reasoning capabilities of Large Language Models (LLMs).

🎉ARI-QA

This is the code for the paper Temporal Knowledge Question Answering via Abstract Reasoning Induction (Chen et al., ACL 2024).

Architecture of ARI-QA

ARI (Abstract Reasoning Induction) is an innovative framework designed to enhance the temporal reasoning capabilities of Large Language Models (LLMs). By dividing the reasoning process into two distinct phases—Knowledge-agnostic and Knowledge-based—ARI aims to mitigate the issues of hallucinations and improve LLMs' ability to integrate abstract methodologies derived from historical data.

Datasets

🤗MultiTQ Datasets Link: https://huggingface.co/datasets/chenziyang/MultiTQ

🤗CronQuestion Datasets Link: https://github.com/apoorvumang/CronKGQA

Example questions	Answer
Who condemned Abhisit Vejjajiva in May 2010?	Thailand
Who was the first to visit the Middle East in 2008?	Frank Bainimarama
When did the Aam Aadmi Party first negotiated with Harish Rawat?	2015-12-13
Who expressed intent to engage in diplomatic cooperation with Ethiopia before Jun 25th, 2006?	China

Dataset used in this paper can be found in ./data folder.

git clone git@github.com:czy1999/ARI-QA.git

cd ./ARI-QA/data
unzip data.zip

Running the code

running jupyter notebook

# For CronQuestions
# run ./CronQuestions/run_cronquestions.ipynb

# For MultiTQ
# run ./MultiTQ/run_multitq.ipynb

Note: Before running the code, make sure to input your OpenAI key in utils.py.

Cite

If you find our method, code, or experimental setups useful, please cite our paper:

@inproceedings{chen-etal-2024-temporal,
    title = "Temporal Knowledge Question Answering via Abstract Reasoning Induction",
    author = "Chen, Ziyang  and
      Li, Dongfang  and
      Zhao, Xiang  and
      Hu, Baotian  and
      Zhang, Min",
    editor = "Ku, Lun-Wei  and
      Martins, Andre  and
      Srikumar, Vivek",
    booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.acl-long.267",
    pages = "4872--4889"
}