Tool Learning Papers

Must-read papers on tool learning with foundation models. We also made some slides about tool learning.

Content

Why Tool Learning?
Keywords Convention
Papers
Contribution
- Contributors
- Contributing to this paper list

Why Tool Learning?

Humans possess an extraordinary ability to create and utilize tools, allowing them to overcome physical limitations and explore new frontiers. With the advent of foundation models, AI systems have the potential to be equally adept in tool use as humans. This paradigm, i.e., tool learning with foundation models, combines the strengths of specialized tools and foundation models to achieve enhanced accuracy, efficiency, and automation in problem-solving. Despite its immense potential, there is still a lack of a comprehensive understanding of key challenges, opportunities, and future endeavors in this field. The field of tool learning is developing fast with many works being proposed lately. To make these works more accessible, we have created this repository with a list of resources related to tool learning, hoping to facilitate the development of this area.

Keywords Convention

The abbreviation of the work.

The utilized tool in the work.

Other important information of the work.

Papers

Survey

Tool Learning with Foundation Models, Preprint 2023.

Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang, Chaojun Xiao, Chi Han, Yi Ren Fung, Yusheng Su, Huadong Wang, Cheng Qian, Runchu Tian, Kunlun Zhu, Shihao Liang, Xingyu Shen, Bokai Xu, Zhen Zhang, Yining Ye, Bowen Li, Ziwei Tang, Jing Yi, Yuzhang Zhu, Zhenning Dai, Lan Yan, Xin Cong, Yaxi Lu, Weilin Zhao, Yuxiang Huang, Junxi Yan, Xu Han, Xian Sun, Dahai Li, Jason Phang, Cheng Yang, Tongshuang Wu, Heng Ji, Zhiyuan Liu, Maosong Sun. [pdf], [Project]
Augmented Language Models: a Survey, Preprint 2023.

Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis, Ram Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, Edouard Grave, Yann LeCun, Thomas Scialom. [pdf]
Foundation Models for Decision Making: Problems, Methods, and Opportunities, Preprint 2023.

Sherry Yang, Ofir Nachum, Yilun Du, Jason Wei, Pieter Abbeel, Dale Schuurmans. [pdf]
Interactive Natural Language Processing, Preprint 2023.

Zekun Wang, Ge Zhang, Kexin Yang, Ning Shi, Wangchunshu Zhou, Shaochun Hao, Guangzheng Xiong, Yizhi Li, Mong Yuan Sim, Xiuying Chen, Qingqing Zhu, Zhenzhu Yang, Adam Nik, Qi Liu, Chenghua Lin, Shi Wang, Ruibo Liu, Wenhu Chen, Ke Xu, Dayiheng Liu, Yike Guo, Jie Fu. [pdf]

Tool-augmented Learning

Toolformer: Language Models Can Teach Themselves to Use Tools Preprint 2023.

Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, Thomas Scialom [pdf], [Code]
Generalization through Memorization: Nearest Neighbor Language Models ICLR 2020.

Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, Mike Lewis [pdf] [Code]
Improving Language Models by Retrieving from Trillions of Tokens PMLR 2022.

Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego De Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack Rae, Erich Elsen, Laurent Sifre [pdf], [blog]
Extracting training data from large language models USENIX 2021.

Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, Colin Raffel [pdf], [code]
Lamda: Language models for dialog applications Preprint 2022.

Aaron Daniel Cohen, Adam Roberts, Aleandra Molina, Alena Butryna, Alicia Jin, Apoorv Kulshreshtha, Ben Hutchinson, Ben Zevenbergen,Blaise Hilary Aguera Arcas, Chung-ching Chang, Claire Cui, Cosmo Du, Daniel De Freitas Adiwardana, Dehao Chen, Dmitry (Dima) Lepikhin, Ed H. Chi-rin Hoffman-ohn. Heno-1ze Chend. Honarae ee. cor krivokon. James 0in. amie Ha. oe Fenton. ohnny Soraker. athy Meier-HelstemKristen 0lson, Lora Mois Aroyo, Maarten Paul Bosma, Marc Joseph Pickett, Marcelo Amorim Menegali, Marian Croak, Mark Diaz, Matthew Lamm,Maxim Krikun, Meredith RingelMoris,Noam Shazeer, Quoc V Le, Rachel Berstein, Ravi Rajakumar, Ray kurzwei, Roma Thoppilan, Steven ZhengTaylor Bos,Toju Duke, Tulse Doshi, Vincent . Zhao, Vinodkumar Prabhakaran, Wil Rusch, YaGuang Li, Yanping Huang, Yangi Zhou, Yuanzhong XuZhifeng Chen [pdf]
Recitation-Augmented Language Models Preprint 2022.

Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, Denny Zhou [pdf], [code]

Re-imagen: Retrieval-augmented text-to-image generator Preprint 2022.

Wenhu Chen, Hexiang Hu, Chitwan Saharia, William W. Cohen [pdf]
Knn-diffusion: Image generation via large-scale retrieval Preprint 2022.

Shelly Sheynin, Oron Ashual, Adam Polyak, Uriel Singer, Oran Gafni, Eliya Nachmani, Yaniv Taigman [pdf]
Training verifiers to solve math word problems Preprint 2021.

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, John Schulman [pdf]
Talm: Tool augmented language models Preprint 2022.

Aaron Parisi, Yao Zhao, Noah Fiedel [pdf]
Mind's Eye: Grounded Language Model Reasoning through Simulation Preprint 2022.

Ruibo Liu, Jason Wei, Shixiang Shane Gu, Te-Yen Wu, Soroush Vosoughi, Claire Cui, Denny Zhou, Andrew M. Dai [pdf]
Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks Preprint 2022.

Wenhu Chen, Xueguang Ma, Xinyi Wang, William W. Cohen [pdf]
PAL: Program-aided Language Models Preprint 2022.

Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, Graham Neubig [pdf]
Behavior Cloned Transformers are Neurosymbolic Reasoners Preprint 2022.

Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté, Prithviraj Ammanabrolu [pdf]
Show your work: Scratchpads for intermediate computation with language models Preprint 2021.

Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, Augustus Odena [pdf]
Making Language Models Better Tool Learners with Execution Feedback Preprint 2023.

Shuofei Qiao, Honghao Gui, Huajun Chen, Ningyu Zhang [pdf]
GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Information Preprint 2023.

Qiao Jin, Yifan Yang, Qingyu Chen, Zhiyong Lu [pdf]
Adaptive Chameleon or Stubborn Sloth: Unraveling the Behavior of Large Language Models in Knowledge Conflicts Preprint 2023.

Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, Yu Su [pdf]

Tool-oriented Learning

WebCPM: Interactive Web Search for Chinese Long-form Question Answering Preprint 2023.

Yujia Qin, Zihan Cai, Dian Jin, Lan Yan, Shihao Liang, Kunlun Zhu, Yankai Lin, Xu Han, Ning Ding, Huadong Wang, Ruobing Xie, Fanchao Qi, Zhiyuan Liu, Maosong Sun and Jie Zhou [pdf], [Code]
CREATOR: Disentangling Abstract and Concrete Reasonings of Large Language Models through Tool Creation Preprint 2023.

Cheng Qian, Chi Han, Yi R. Fung, Yujia Qin, Zhiyuan Liu, Heng Ji [pdf], [Code]
Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents Preprint 2023.

Zihao Wang, Shaofei Cai, Anji Liu, Xiaojian Ma, Yitao Liang [Project], [pdf], [Code]
On the Tool Manipulation Capability of Open-source Large Language Models Preprint 2023.

Qiantong Xu, Fenglu Hong, Bo Li, Changran Hu, Zhengyu Chen, Jian Zhang [pdf], [Code]
WebGPT: Browser-assisted question-answering with human feedback Preprint 2023.

Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, John Schulman [pdf], [Blog]
ReAct: Synergizing Reasoning and Acting in Language Models ICLR 2023.

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao [pdf], [Blog], [Code]
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents PMLR, 2022.

Wenlong Huang, Pieter Abbeel, Deepak Pathak, Igor Mordatch [pdf], [Blog], [Code]
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances Preprint 2022.

Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil J Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Kuang-Huei Lee, Sergey Levine, Yao Lu, Linda Luu, Carolina Parada, Peter Pastor, Jornell Quiambao, Kanishka Rao, Jarek Rettinghouse, Diego Reyes, Pierre Sermanet, Nicolas Sievers, Clayton Tan, Alexander Toshev, Vincent Vanhoucke, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Mengyuan Yan, Andy Zeng [pdf], [Blog], [Code]
Inner Monologue: Embodied Reasoning through Planning with Language Models Preprint 2022.

Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, Pierre Sermanet, Noah Brown, Tomas Jackson, Linda Luu, Sergey Levine, Karol Hausman, Brian Ichter [pdf], [Blog]
Code as Policies: Language Model Programs for Embodied Control Preprint 2022.

Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, Andy Zeng [pdf], [Blog], [Code]
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models Preprint 2022.

Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, Animesh Garg [pdf]
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language Preprint 2022.

Andy Zeng, Maria Attarian, Brian Ichter, Krzysztof Choromanski, Adrian Wong, Stefan Welker, Federico Tombari, Aveek Purohit, Michael Ryoo, Vikas Sindhwani, Johnny Lee, Vincent Vanhoucke, Pete Florence [pdf]
A Generalist Agent Preprint 2022.

Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordbar, Nando de Freitas [pdf]
PaLM-E: An Embodied Multimodal Language Model Preprint 2023.

Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, Pete Florence [pdf], [Blog]
WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents Preprint 2022.

Shunyu Yao, Howard Chen, John Yang, Karthik Narasimhan [pdf], [Blog], [Code]
Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models Preprint 2023.

Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, Nan Duan [pdf], [Code]
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace Preprint 2023.

Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, Yueting Zhuang [pdf], [Code]
Language Models can Solve Computer Tasks Preprint 2023.

Geunwoo Kim, Pierre Baldi, Stephen McAleer [pdf]
ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings Preprint 2023.

Shibo Hao, Tianyang Liu, Zhen Wang, Zhiting Hu [pdf]