Code-it is simultaneously:
- A standalone package to generate code and execute with local LLMs.
- An importable tool into langchain
This is a highly experimental project, the quality of the generations may not be high enough for production usage.
Code-it leverages LLMs to generate code - unlike other solutions, it doesn't try to rely on the smartness of LLMs, but rather assume they are rather dumb and perform several mistakes along the way. It applies a simple algorithm to iteratively code towards it's task objective, in a similar way a programmer might do. This algorithm is implemented with control statements and different prompts to steer the LLM at performing the correct action.
It is not an autonomous agent - at most, we could call it semi-autonomous.
- Setup https://github.com/oobabooga/text-generation-webui with API enabled
- Install it through pip / git on your project. For, you can define this line in your project requirements.txt:
code_it @ git+https://github.com/paolorechia/code-it
Note that I did not yet have tags or a PyPi package, as I'm not sure how useful this package will be in the future.
- Locally as a standalone program with your current Python shell / virtualenv:
git clone https://github.com/paolorechia/code-it
cd code-it
pip install -r requirements.txtWARNING: the LLM will run arbitrary code, use it at your own risk.
Execute the main:
python3 -m code_it
This will save the code in persistent_source.py
Change the task in the task.txt file to perform another task.
You can reuse the code from https://github.com/paolorechia/code-it/blob/main/code_it/__main__.py
Here's the base minimum code to use this library:
from code_it.code_editor.python_editor import PythonCodeEditor
from code_it.models import build_text_generation_web_ui_client_llm, build_llama_base_llm
from code_it.task_executor import TaskExecutor, TaskExecutionConfig
code_editor = PythonCodeEditor()
model_builder = build_llama_base_llm
config = TaskExecutionConfig()
task_executor = TaskExecutor(code_editor, model_builder, config)
with open("task.txt", "r") as fp:
task = fp.read()
task_executor.execute(task)Here we import the PythonCodeEditor, currently the only supported editor, along with a llama LLM. Notice that this assumes a server running on 0.0.0.0:8000, which comes from my other repo: https://github.com/paolorechia/learn-langchain/blob/main/servers/vicuna_server.py
You can easily change this to instead use the text-generation-web-ui tool from oobagooba, by importing the builder: build_text_generation_web_ui_client_llm. Implementing your own model client should also be straightforward. Look at the source code in: https://github.com/paolorechia/code-it/blob/main/code_it/models.py
Notice that in the example above we imported the TaskExecutionConfig, let's look at this class:
@dataclass
class TaskExecutionConfig:
execute_code = True
install_dependencies = True
apply_linter = True
check_package_is_in_pypi = True
log_to_stdout = True
coding_samples = 3
code_sampling_strategy = "PYLINT"
sampling_temperature_multipler = 0.1
dependency_samples = 3
max_coding_attempts = 5
dependency_install_attempts = 5
planner_temperature = 0
coder_temperature = 0
linter_temperature = 0.3
dependency_tracker_temperature = 0.2You can change these parameters to change how the program behaves. Not all settings are always applied at the same time, for instance, if you change the code_sampling_strategy to NO_SAMPLING, then of course the config parameter sampling_temperature_multiplier is not used.
To understand these settings better, you should read the task execution code directly, as there is no detailed documentation for this yet: https://github.com/paolorechia/code-it/blob/main/code_it/task_executor.py
Here's an example using my other repo: https://github.com/paolorechia/learn-langchain/blob/main/langchain_app/executor_tests/chuck_norris_joke.py
from langchain.agents import initialize_agent, AgentType
from langchain_app.models.vicuna_request_llm import VicunaLLM
from code_it.models import build_llama_base_llm
from code_it.langchain.code_it_tool import CodeItTool
from code_it.task_executor import TaskExecutionConfig
llm = VicunaLLM()
config = TaskExecutionConfig()
print(config)
config.install_dependencies = True
config.execute_code = True
code_editor = CodeItTool(build_llama_base_llm, config)
tools = [
code_editor.build_execute_task(),
]
agent = initialize_agent(
tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True
)
agent.run(
"""
Remember to use the following format:
Action: <>
Action Input:
<>
Question: Extract a joke from https://api.chucknorris.io/jokes/random - access the key 'value' from the returned JSON.
"""
)The Mixin gives the option to use the pip install command from the code_it virtualenv manager, effectively adding package installation powers to your LLM inside langchain.
Note that the Mixin does not work as well as the task execution tool.
The local models quite often fails to use the new actions appropriately, so even this example does not yet work as you would expect.
rom langchain.agents import (
AgentExecutor,
LLMSingleActionAgent,
Tool,
AgentOutputParser,
)
from langchain.prompts import StringPromptTemplate
from langchain import LLMChain
from langchain_app.models.vicuna_request_llm import VicunaLLM
from langchain.schema import AgentAction, AgentFinish
from code_it.langchain.python_langchain_tool_mixin import LangchainPythonToolMixin
import re
from typing import List, Union
llm = VicunaLLM()
code_editor = LangchainPythonToolMixin()
tools = [
code_editor.build_add_code_tool(),
code_editor.build_run_tool(),
code_editor.build_pip_install()
]
template = """You're a programmer AI.
You are asked to code a certain task.
You have access to a Code Editor, that can be used through the following tools:
{tools}
You should ALWAYS think what to do next.
Use the following format:
Task: the input task you must implement
Current Source Code: Your current code state that you are editing
Thought: you should always think about what to code next
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: The result of your last action
... (this Thought/Action/Action Input/Source Code/Code Result can repeat N times)
Thought: I have finished the task
Task Completed: the task has been implemented
Example task:
Task: the input task you must implement
Thought: To start, we need to add the line of code to print 'hello world'
Action: CodeEditorAddCode
Action Input:
print("hello world") end of llm ouput
Observation:None
Thought: I have added the line of code to print 'hello world'. I should execute the code to test the output
Action: CodeEditorRunCode
Action Input:
Observation:Program Succeeded
Stdout:b'hello world\n'
Stderr:b''
Thought: The output is correct, it should be 'hello world'
Action: None
Action Input:
Output is correct
Observation:None is not a valid tool, try another one.
Thought: I have concluded that the output is correct
Task Completed: the task is completed.
REMEMBER: don't install the same package more than once
Now we begin with a real task!
Task: {input}
Source Code: {source_code}
{agent_scratchpad}
Thought:"""
# Set up a prompt template
class CodeEditorPromptTemplate(StringPromptTemplate):
# The template to use
template: str
code_editor: LangchainPythonToolMixin
tools: List[Tool]
def format(self, **kwargs) -> str:
# Get the intermediate steps (AgentAction, Observation tuples)
# Format them in a particular way
intermediate_steps = kwargs.pop("intermediate_steps")
thoughts = ""
for action, observation in intermediate_steps:
thoughts += action.log
thoughts += f"\nObservation: {observation}\nThought: "
# Set the agent_scratchpad variable to that value
kwargs["agent_scratchpad"] = thoughts
kwargs["source_code"] = code_editor.display_code()
kwargs["tools"] = "\n".join(
[f"{tool.name}: {tool.description}" for tool in self.tools]
)
kwargs["tool_names"] = ", ".join([tool.name for tool in self.tools])
return self.template.format(**kwargs)
prompt = CodeEditorPromptTemplate(
template=template,
code_editor=code_editor,
tools=tools,
input_variables=["input", "intermediate_steps"],
)
class CodeEditorOutputParser(AgentOutputParser):
def parse(self, llm_output: str) -> Union[AgentAction, AgentFinish]:
print("llm output: ", llm_output, "end of llm ouput")
# Check if agent should finish
if "Task Completed:" in llm_output:
return AgentFinish(
# Return values is generally always a dictionary with a single `output` key
# It is not recommended to try anything else at the moment :)
return_values={"output": llm_output},
log=llm_output,
)
# Parse out the action and action input
regex = r"Action\s*\d*\s*:(.*?)\nAction\s*\d*\s*Input\s*\d*\s*:[\s]*(.*)"
match = re.search(regex, llm_output, re.DOTALL)
if not match:
raise ValueError(f"Could not parse LLM output: `{llm_output}`")
action = match.group(1).strip()
action_input = match.group(2)
# Return the action and action input
return AgentAction(
tool=action, tool_input=action_input.strip(" ").strip('"'), log=llm_output
)
output_parser = CodeEditorOutputParser()
llm_chain = LLMChain(llm=llm, prompt=prompt)
llm = VicunaLLM()
tool_names = [tool.name for tool in tools]
agent = LLMSingleActionAgent(
llm_chain=llm_chain,
output_parser=output_parser,
stop=["\nObservation:"],
allowed_tools=tool_names,
)
agent_executor = AgentExecutor.from_agent_and_tools(
agent=agent, tools=tools, verbose=True
)
agent_executor.run(
"""
Your job is to plot an example chart using matplotlib. Create your own random data.
Run this code only when you're finished.
DO NOT add code and run into a single step.
"""
)