About evaluation method

[Hannibal046]

Hi, Thanks for the great work. When I using the sample evaluation scripts, I got the following error. Could you please help me out?

[yhoshi3]

Thank you for trying our tool and bringing the issue to our attention.
We found an error in the evaluation config, and we have now rectified it.

"e5-8b_flat" -> "flat_subset_499992"

The error you mentioned occurred because the specified retriever name in the evaluation config did not exist in the retriever config.

After the embedding process and construction of the Faiss index are complete, retrieval can be performed by correctly specifying the path to the built index in the retriever config (cf. indexing.md).

If you find indexing the corpus with the e5 model to be inconvenient, you can opt for BM25 indexing, which requires less time.

I hope this helps.

[Hannibal046]

Thanks for response! It now works fine!