Make public evaluation code

Question

Make public evaluation code

Closed this issue 5 months ago · 5 comments

Hi, would you please consider making public the code to reproduce the results in your paper? Thanks!

Answer 1 · 2024-03-28T09:26:22.000Z

Hi, the evaluation code has already been made public. Please check the evaluate.py.

Answer 2 · 2024-03-28T19:41:32.000Z

@yixuantt I believe evaluate.py evaluates only the retrieval results, not the final task itself (question answering accuracy from Table 6).

Answer 3 · 2024-04-01T22:41:57.000Z

@hugoabonizio Hi Hugo, you can check the qa_llama.py, which I updated. This is a demo script of my question-answering process.

Answer 4 · 2024-06-25T23:10:09.000Z

@hugoabonizio Hi, how did you calculate the Accuracy for Table 6? Could you provide a formula? just estimate whether the gold-answer is in the model_answer ?

Answer 5 · 2024-08-02T02:39:08.000Z

closed as the evaluation code has been public.