/ReAct

[ICLR 2023] ReAct: Synergizing Reasoning and Acting in Language Models

Primary LanguageJupyter NotebookMIT LicenseMIT

ReAct Prompting

GPT-3 prompting code for ICLR 2023 paper ReAct: Synergizing Reasoning and Acting in Language Models.

To use ReAct for more tasks, consider trying LangChain's zero-shot ReAct Agent.

Setup

You need to first have an OpenAI API key and store it in the environment variable OPENAI_API_KEY (see here).

Package requirement: openai, and install alfworld following instructions here.

Experiments

Run {hotpotqa,fever,alfworld,webshop}.ipynb. As HotpotQA and FEVER have large validation sets, we only run 500 random examples (see notebooks). We find PaLM and GPT-3 are better at different tasks.

HotpotQA (500 random dev, EM) FEVER (500 random dev, EM) AlfWorld (success rate) WebShop (success rate)
PaLM-540B (paper) 29.4 62.2 70.9 40
GPT-3 (davinci-002) 30.4 54 78.4 35.8

Citation

@inproceedings{yao2023react,
  title = {{ReAct}: Synergizing Reasoning and Acting in Language Models},
  author = {Yao, Shunyu and Zhao, Jeffrey and Yu, Dian and Du, Nan and Shafran, Izhak and Narasimhan, Karthik and Cao, Yuan},
  booktitle = {International Conference on Learning Representations (ICLR) },
  year = {2023},
  html = {https://arxiv.org/abs/2210.03629},
}