aws/fmeval

Evaluation Algorithm for Recommendations

mccartni-aws opened this issue · 4 comments

Thanks for the great work.

I'm working with a customer that is looking to use LLMs for recommendation. We are proactively working on some evaluation metrics/algorithms.

Are there any plans to add any evaluation algorithms for recommendations?
I am happy to contribute and submit a PR for this, if not.

Hi @mccartni-aws, thanks for reaching out!

An upcoming release will contain changes that make contributing to our codebase a much smoother experience, so if you aren't in a rush, it will be a more pleasant experience for you if you wait until then. However, contributions are always welcome.

Hey @danielezhu
That's fine, I can wait until then 🦖