SELFGOAL: Your Language Agents Already Know How to Achieve High-level Goals

SELFGOAL

SELFGOAL is a non-parametric learning algorithm for language agents, i.e., without parameter update. Concretely, SELFGOAL is featured with two key modules, Decomposition and Search which construct and utilize a subgoal tree respectively, namely GoalTree, to interact with the environment.

More details are in the paper.

Setup Environment

Create a conda environment and install dependency:

conda create -n selfgoal python=3.11
conda activate selfgoal
pip install -r requirements.txt

Here, we use AucArena as an example to illustrate the Decompose and Search Modules and how the agent utilizes guidance to make decisions.

Decompose Module

To adapt SELFGOAL to your current scenario, you have to describe your environment in prompt_base.py

Main Goal Decomposition

# main goal
Imagine you are an agent in a {scene}. 

Taking an analogy from human behaviors, if your fundamental objective in this scenario is "{goal}", what sub-goals you might have?

Sub-Goal Decomposition

# sub-goals
{scene}

---

For the goal: "{sub_goal}", based on the current state, can you further run some deduction for fine-grained goals or brief guidelines?

Search Module

At each stage, you must also supply SELFGOAL with the current state to enable the Search Module to identify the most useful sub-goals.

# Sub-Goal
Here's the current scenario:

{scene}

------------------------------
For the goal: "{sub_goal}", can you further run some deduction for fine-grained goals or brief guidelines?

An example of guidance

The generated sub-goal will be displayed as shown in the following example:

Based on the current auction scenario, here are some derived sub-goals and detailed guidance to help you better achieve your primary objective, i.e., maximize your total profit: 
* Sub-goal: Market Value Research: Understand the current market value of the item being auctioned. This will help you estimate the potential profit and decide if the bidding price is justified.
* Sub-goal: Profit Margin Calculation: Calculate the potential profit margin based on the current bidding price and the market value of the item. This will help you understand if the cost of bidding is worth the potential benefit.

Please consider these sub-goals and detailed advice in your next round of strategic planning and action execution in the auction to help achieve your primary objective.

Act Module

To prompt the agent to use the selected sub-goals in its decision-making process, incorporate the guidance into the system message.

Here are some possible subgoals and guidance derived from your primary objective:

{sub-goals}

In this round, You may target some of these subgoals and guidance to improve your bidding strategy and action, in order to achieve your primary objective.

You should also prompt agents in each round of decision-making, for example, using Reflect on derived sub-goals.

Run

For Open Source Models

For open source models, deploy the model as an API using vLLM

python -m vllm.entrypoints.openai.api_server --model facebook/opt-125m --port 8001

AucArena

python auction_workflow.py --shuffle --repeat 10 -t 4

DealOrNotDeal

python gpt_bargain.py

GAMABench

python guessing_game.py
python public_goods.py

Contact

If you have any problems, please contact Ruihan Yang.

Citation

@article{yang2024selfgoallanguageagentsknow,
      title={SelfGoal: Your Language Agents Already Know How to Achieve High-level Goals}, 
      author={Ruihan Yang and Jiangjie Chen and Yikai Zhang and Siyu Yuan and Aili Chen and Kyle Richardson and Yanghua Xiao and Deqing Yang},
      year={2024},
      eprint={2406.04784},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2406.04784}, 
}

rhyang2021/SELFGOAL