agential-ai/agential

[Feature Request]: Standardize All Prompt-based Techniques

Closed this issue · 0 comments

Feature Description

All the prompt-based agent methods should be standardized:

  • k=5 fewshot examples (later we can also make k=1 fewshot examples)
  • across all methods for each benchmark, use the same k-shot examples
  • each benchmark should have that benchmark's specific fewshot examples
  • you will need to, both truncate/pad the fewshot examples to k and also ensure the k fewshot examples for a benchmark is consistent across all methods (at least methods that already have fewshot examples for them)

Reason

No response