Question about the pre-given passages
WenzhengZhang opened this issue · 1 comments
Hi @AkariAsai, great work and thanks for this repo!
I'm confused about the pre-given passages in the evaluation datasets. Are they retrieved using the initial question as query or you took the union of all the retrieved passages at multiple stages?
Will you release the non-static inference code without pre-given passages or the code for generating the pre-given passages for the evaluation datasets?
For the static inference, the retrieved passages are only retrieved at the beginning. Note that for ALCE ASQA, we use the pre-given passages provided by the original repository, which were initially retrieved by GTR.
I apologize for the delay of releasing online retrieval support. I am only one who maintain this repository, and I've been hectic with teaching duties and other projects as a Ph.D. student in the past two quarters... I'm trying to improve the code bases by ICLR.