emrgnt-cmplxty/automata

Refine the available `AutomataAgents`

Closed this issue · 0 comments

Right now the production agent pipeline offers the three following agents

  • automata-main - The main agent which carries out high-level planning and delegates tasks to sub-agents
  • automata-writer - A helper agent responsible for writing code into output files. Future responsibilities could be testing and iterating on said code.
  • automata-reader - A helper agent responsible which parses natural language queries to retrieve relevant code context from the local repository.

When provided with the right tools, we predict that three agents can reach proficiency in basic programming tasks. The most importing supporting tools for this objective are the ContextOracle and the PyWriter.

The goal of this issue is to refine our approach to get us closer to this objective. These agents need to be tested and refreshed with the latest tooling to bring us to this objective. Thus, the task is to identify some key tasks which can be ran to assess the agent performance, and to iterate until we believe we are closer to optimal performance. Over time a task such as this should become more quantitative, but for now we will adopt an ad-hoc approach in order to bolster our demo tape.

As always, don't hesitate to ask if you have any questions or need further clarification. Your contributions to this project are highly valued!