FSoft-AI4Code/AgileCoder

Unable to Generate a Software w/ AgileCoder

diopresi opened this issue · 4 comments

diopresi commented

Description:

I encountered multiple issues while trying to create a software project using AgileCoder.

Steps to Reproduce:

  1. Using the Command-Line Interface from PyPi:

    • Run the command:

      agilecoder --task "create a tic-tac-toe game in python"

    • During execution, I faced an issue related to tree-sitter-python.

    • I attempted to resolve this by installing the package as suggested in the error message:

      pip install git+https://github.com/tree-sitter/tree-sitter-python.git@master

    • After retrying, I encountered the following errors:

      /Users/admin/Desktop/git/augest/poc/agile-coder-poc-off/venv/lib/python3.9/site-packages/urllib3/__init__.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020
  warnings.warn(
------------------------------
/Users/admin/Desktop/git/augest/poc/agile-coder-poc-off/venv/lib/python3.9/site-packages/nltk/translate/bleu_score.py:552: UserWarning:
The hypothesis contains 0 counts of 4-gram overlaps.
Therefore the BLEU score evaluates to 0, independently of
how many N-gram overlaps of lower order it contains.
Consider using lower n-gram order or use SmoothingFunction()
  warnings.warn(_msg)
Traceback (most recent call last):
  File "/Users/admin/Desktop/git/augest/poc/agile-coder-poc-off/venv/bin/agilecoder", line 8, in <module>
    sys.exit(main())
  File "/Users/admin/Desktop/git/augest/poc/agile-coder-poc-off/venv/lib/python3.9/site-packages/agilecoder/cli.py", line 24, in main
    run_task(args)
  File "/Users/admin/Desktop/git/augest/poc/agile-coder-poc-off/venv/lib/python3.9/site-packages/agilecoder/run_api.py", line 114, in run_task
    chat_chain.execute_chain()
  File "/Users/admin/Desktop/git/augest/poc/agile-coder-poc-off/venv/lib/python3.9/site-packages/agilecoder/components/chat_chain.py", line 165, in execute_chain
    self.execute_step(phase_item)
  File "/Users/admin/Desktop/git/augest/poc/agile-coder-poc-off/venv/lib/python3.9/site-packages/agilecoder/components/chat_chain.py", line 154, in execute_step
    self.chat_env = compose_phase_instance.execute(self.chat_env)
  File "/Users/admin/Desktop/git/augest/poc/agile-coder-poc-off/venv/lib/python3.9/site-packages/agilecoder/components/composed_phase.py", line 148, in execute
    self.phases[phase].update_phase_env(chat_env)
  File "/Users/admin/Desktop/git/augest/poc/agile-coder-poc-off/venv/lib/python3.9/site-packages/agilecoder/components/phase.py", line 600, in update_phase_env
    all_done_tasks = '\n'.join(chat_env.env_dict['done-works'])
KeyError: 'done-works'

  2. Running Directly from Source Code:

    • I also tried running the project directly from the source code and encountered the following error:

      /Users/admin/Desktop/git/augest/poc/AgileCoder/venv/lib/python3.9/site-packages/urllib3/__init__.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020
  warnings.warn(
------------------------------
Traceback (most recent call last):
  File "/Users/admin/Desktop/git/augest/poc/AgileCoder/venv/bin/agilecoder", line 33, in <module>
    sys.exit(load_entry_point('agilecoder', 'console_scripts', 'agilecoder')())
  File "/Users/admin/Desktop/git/augest/poc/AgileCoder/agilecoder/cli.py", line 24, in main
    run_task(args)
  File "/Users/admin/Desktop/git/augest/poc/AgileCoder/agilecoder/run_api.py", line 114, in run_task
    chat_chain.execute_chain()
  File "/Users/admin/Desktop/git/augest/poc/AgileCoder/agilecoder/components/chat_chain.py", line 165, in execute_chain
    self.execute_step(phase_item)
  File "/Users/admin/Desktop/git/augest/poc/AgileCoder/agilecoder/components/chat_chain.py", line 135, in execute_step
    self.chat_env = self.phases[phase].execute(self.chat_env,
  File "/Users/admin/Desktop/git/augest/poc/AgileCoder/agilecoder/components/phase.py", line 310, in execute
    self.chatting(chat_env=chat_env,
  File "/Users/admin/Desktop/git/augest/poc/AgileCoder/agilecoder/components/utils.py", line 110, in wrapper
    return func(*args, **kwargs)
  File "/Users/admin/Desktop/git/augest/poc/AgileCoder/agilecoder/components/phase.py", line 157, in chatting
    if isinstance(user_response.msg, ChatMessage):
  File "/Users/admin/Desktop/git/augest/poc/AgileCoder/agilecoder/camel/agents/chat_agent.py", line 53, in msg
    raise RuntimeError("error in ChatAgentResponse, info:{}".format(str(self.info)))
RuntimeError: error in ChatAgentResponse, info:{'id': None, 'usage': None, 'termination_reasons': ['max_tokens_exceeded_by_camel'], 'num_tokens': 4125}

Additional Context: I am running this inside a virtual environment. Please advise on potential fixes or workarounds.

Expected Behavior:
The command should execute without errors and generate the tic-tac-toe game.

Actual Behavior:
The command fails with a KeyError: 'done-works' and other warnings when using the CLI. Running directly from the source code results in a RuntimeError due to exceeding the maximum token limit.

minhngh commented

Hi diopres,

Thank you for your feedback. I'm happy to address your concern.

  • Regarding the error: KeyError: 'done-works': we speculate there is a potential issue with the installation, which may lead to this bug. To resolve this error, we recommend that you create a new environment and then reinstall by running
    pip install agilecoder
    pip install git+https://github.com/tree-sitter/tree-sitter-python.git@master
    Be sure to check whether tree-sitter-python is installed successully by testing the statement: import tree_sitter_python
    If you still have the problem, please provide more details about your system (OS, python version).
  • Regarding the problem with the source code: In fact, the source code on the master branch is obsolete, so there may be some problems with this branch. The latest source code is on the new-flow branch, but it should be noted that it hasn't been tested carefully. If you want to run from source code, you can clone the new-flow branch at the 4a1b863 commit.

If you have any concerns, don't hesitate to reach out to us.

svanschalkwyk commented

checked out new-flow, pip install -e AgileCoder.
"Successfully installed agilecoder-0.2.0 codebleu-0.7.0 google-auth-oauthlib-1.2.1 markdown-3.5.2 nltk-3.8.1 oauthlib-3.2.2 pygame-2.6.0 requests-oauthlib-2.0.0 tenacity-8.2.3 tiktoken-0.6.0 tree-sitter-0.22.3" so tree-sitter is there.

import tree_sitter_python

ModuleNotFoundError: No module named 'tree_sitter_python'

pip install git+https://github.com/tree-sitter/tree-sitter-python.git@master seems to work

diopresi commented

Hi @minhngh,

The issues persist despite trying several approaches. Here’s a summary of what I’ve attempted so far:

Using PyPi Installation:

  • Set up a new virtual environment and installed packages via pip install agilecoder and pip install git+https://github.com/tree-sitter/tree-sitter-python.git@master.
  • Verified the tree-sitter installation by importing tree_sitter_python, which was successful.
  • Encountered the same error as @svanschalkwyk mentioned in #5 (AttributeError: 'NoneType' object has no attribute 'group' from extract_pl_framework_text).
  • The issue occurs regardless of the model (tried with OpenAI GPT-4 and GPT-4 on Azure).
  • Python Versions Tried: 3.9, 3.10, 3.11, 3.12
  • OS: macOS 12.6.8

Using Source Code:

  • Ran the code from commit 4a1b863 and the latest commit from new-flow.
  • Encountered the same issue as #5, with consistent results across multiple attempts.
  • Python Versions Tried: 3.9, 3.10, 3.11, 3.12
  • OS: macOS 12.6.8

On both case, When passing the --model parameter with GPT_4, the following error occurs:

File "/AgileCoder/agilecoder/camel/agents/chat_agent.py", line 53, in msg
    raise RuntimeError("error in ChatAgentResponse, info:{}".format(str(self.info)))
RuntimeError: error in ChatAgentResponse, info:{'id': None, 'usage': None, 'termination_reasons': ['max_tokens_exceeded_by_camel'], 'num_tokens': 4123}

Any insights or further steps to resolve this issue would be appreciated. If you don't mind could you share the baseline requirement to run it?

Hi diopres,

Thank you for your feedback. I'm happy to address your concern.

  • Regarding the error: KeyError: 'done-works': we speculate there is a potential issue with the installation, which may lead to this bug. To resolve this error, we recommend that you create a new environment and then reinstall by running
    pip install agilecoder
    pip install git+https://github.com/tree-sitter/tree-sitter-python.git@master
    Be sure to check whether tree-sitter-python is installed successully by testing the statement: import tree_sitter_python
    If you still have the problem, please provide more details about your system (OS, python version).
  • Regarding the problem with the source code: In fact, the source code on the master branch is obsolete, so there may be some problems with this branch. The latest source code is on the new-flow branch, but it should be noted that it hasn't been tested carefully. If you want to run from source code, you can clone the new-flow branch at the 4a1b863 commit.

If you have any concerns, don't hesitate to reach out to us.

minhngh commented

Hi diopres,

Thank you for your feedback. Below are our responses:

  • Using Pypi:
    We've tried to reproduce the error by following your described steps with some models, including GPT-4o, GPT-3.5, and Claude Haiku as we don't have GPT-4 key, but we haven't encountered this bug. Our steps are as below:
    1. conda deactivate; conda create -n agilecoder python=3.11 (I prefer python 3.11 as it is faster than its predecessors)
    2. conda activate agilecoder
    3. pip install agilecoder
    4. pip install git+https://github.com/tree-sitter/tree-sitter-python.git@master

You can try following our steps. If the error still persists, our library actually logs each run in a file in the folder ~/WareHouse/. Can you provide log files with erroneous runs for deeper analysis?

  • Using Source code:
    We currently set a pretty limited context length, 4K for GPT-3.5 and 8K for GPT-4. Unfortunately, we don't support to adjust this threshold now. In addition, we acknowledge that although our proposed graph helps to mitigate redundant information in prompts, our method may exceed context length limit. However, the task you used is not very complicated, so can you provide corresponding log files?