[Feature Request]: Universal Selector
Closed this issue · 0 comments
alckasoc commented
Feature Description
We have a function, get_fewshot_examples,
which automatically selects the few-shot examples in manager.py
based on benchmark type, name, and few-shot type. We also have strategy_factory.py
, which automatically selects the correct strategy based on benchmark type and name for a specific agent.
We can take this one step further: have a selector that, given a benchmark type/name/few-shot type and agent name, we can select all the relevant examples/prompts for that agent (basically the parameters for generate
). This high-level universal/umbrella selector will simplify high-level usage and imports.
Reason
No response