IBM/unitxt

Allow dynamically created tasks in llm_as_judge

Opened this issue · 0 comments

Currently the task attribute of LLMAsJudgeBase is a string that accepts either the task suffix (for LLMAsJudge) or the task full path (for TaskBasedLLMAsJudge).
Need to change its type to Task and initialize with either the full task path or a Task object, as done with the template attribute of the same class.
This requires a small but non backward compatible change in LLMAsJudge.