Retrieval Tasks Evaluation
Closed this issue · 3 comments
I have noticed that only input data are released in the test set of retrieval tasks.
So what steps should I take if I want to evaluate my own model in this task without the corresponding answer?
May I use the execEval engine to check if the retrieved candidates can pass the unit test?
I would really appreciate it if you could answer this.
No need to use the ExecEval. We will release the gold labels for the retrieval corpus. Our lead author for retrieval task was not available during the final submission. To mitigate the errors, we decided to not release the retrieval gold data. We will release the data shortly. In the meantime, if you need it urgently, please reach out via email.
Thanks for your kind response.
Looking forward to your further update.
Good luck.
Hello, have the gold labels of the nl_code retrieval dataset been released?