agential-ai/agential

[Feature Request]: MBPP for ReAct

Closed this issue · 1 comments

alckasoc commented

Feature Description

Familiarize yourself with the repository and take a look at the ReAct repo, paper, and MBPP.

Currently, the ReAct implementation only has prompts for HotpotQA and FEVER.

Add relevant prompts and logic to the current ReAct implementation. You'll see that an agent's current structure is divided into cog/agent, cog/modules, and cog/functional/. This task will require you to make modifications in cog/prompts, but will also require you to test your code in all the other relevant modules cog/functional and cog/agent.

What to submit:

  • Set up your environment via the CONTRIBUTING.md
  • Make a Pull Request (PR)
  • Add the prompts for the specified benchmark
  • Write a short notebook tmp.ipynb in cog/agent showcasing the agent ran on a sample question from the benchmark
    • Add print statements for all calls to the LLM for easier debugging + I can easily verify the outputs
  • Include a thorough description of your changes within the PR
  • Request a review from @alckasoc

ReAct may not have been tested on this benchmark. If this is true, refer to other methods that have been tested on this benchmark. Check the project lifecycle document.
If there is any additional logic for testing ReAct on this benchmark, include these specifications in the PR description.

Feel free to ask me questions on Slack if you're confused! Good luck!

IGiotto12 commented

I will take this PR.