panruotong/CAG

Lack Dataset ASQA

Closed this issue · 1 comments

HU-xiaobai commented

Hello, excellent research and perfect open-source code!

I would like to ask, it seems the github does not have the ASQA Dataset? Does that means you do not push the implement of ASQA in this code github? Because I see the ASQA results in your paper. Or could I ask how long you will push the implement code of ASQA?

Thanks for your reading and time!

panruotong commented

Hello, thank you for your suggestions.
Our evaluation on the ASQA dataset is based on ALCE. We have now uploaded the dataset with credibility annotations and the corresponding configuration files. You can add our configuration files to the appropriate locations based on the ALCE project and run the provided run.sh script to complete the evaluation for retrieval and credibility. Following the configuration files provided in ALCE, you can also conduct evaluations under the retrieval and retrieval + reranking settings.