Retrieval Speed

Question

Retrieval Speed

Closed this issue 10 months ago · 1 comments

YueYANG1996 commented 10 months ago

Thanks for the great work, it is super useful!

How long does it take to retrieve documents for one query using PubMed as the database? It seems to take me very long. Is there any way to improve the efficiency?

Thank you!

Yue

Answer 1 · 2024-03-21T18:32:10.000Z

I didn't measure this. But it may take around 3 minutes if you use the combination of all corpora (MedCorp) and all retrievers (RRF-4) on A100 GPUs for an end-to-end RAG. It would be much quicker if you use single retrievers.

Using GPU can accelerate this process for sure. For the retrieval phase, you may also replace L82 and L84 in src/utils.py with some approximate indexer in faiss for efficient similarity search.