5 of the available layouts. New layouts are easy to hardcode or generate programmatically.
Overcooked-AI is a benchmark environment for fully cooperative human-AI task performance, based on the wildly popular video game Overcooked.
The goal of the game is to deliver soups as fast as possible. Each soup requires placing up to 3 ingredients in a pot, waiting for the soup to cook, and then having an agent pick up the soup and delivering it. The agents should split up tasks on the fly and coordinate effectively in order to achieve high reward.
You can try out the game here (playing with some previously trained DRL agents). To play with your own trained agents using this interface, you can use this repo. To run human-AI experiments, check out this repo. You can find some human-human and human-AI gameplay data already collected here.
DRL implementations compatible with the environment are included in the repo as a submodule under src/human_aware_rl.
The old human_aware_rl is being deprecated and should only used to reproduce the results in the 2019 paper: On the Utility of Learning about Humans for Human-AI Coordination (also see our blog post).
For simple usage of the environment, it's worthwhile considering using this environment wrapper.
- Carroll, Micah, Rohin Shah, Mark K. Ho, Thomas L. Griffiths, Sanjit A. Seshia, Pieter Abbeel, and Anca Dragan. "On the utility of learning about humans for human-ai coordination." NeurIPS 2019.
- Charakorn, Rujikorn, Poramate Manoonpong, and Nat Dilokthanakul. “Investigating Partner Diversification Methods in Cooperative Multi-Agent Deep Reinforcement Learning.” Neural Information Processing. ICONIP 2020.
- Knott, Paul, Micah Carroll, Sam Devlin, Kamil Ciosek, Katja Hofmann, Anca D. Dragan, and Rohin Shah. "Evaluating the Robustness of Collaborative Agents." AAMAS 2021.
- Nalepka, Patrick, Jordan P. Gregory-Dunsmore, James Simpson, Gaurav Patil, and Michael J. Richardson. "Interaction Flexibility in Artificial Agents Teaming with Humans." Cogsci 2021.
- Fontaine, Matthew C., Ya-Chuan Hsu, Yulun Zhang, Bryon Tjanaka, and Stefanos Nikolaidis. “On the Importance of Environments in Human-Robot Coordination” RSS 2021.
- Zhao, Rui, Jinming Song, Hu Haifeng, Yang Gao, Yi Wu, Zhongqian Sun, Yang Wei. "Maximum Entropy Population Based Training for Zero-Shot Human-AI Coordination". NeurIPS Cooperative AI Workshop, 2021.
- Sarkar, Bidipta, Aditi Talati, Andy Shih, and Dorsa Sadigh. “PantheonRL: A MARL Library for Dynamic Training Interactions”. AAAI 2022.
- Ribeiro, João G., Cassandro Martinho, Alberto Sardinha, Francisco S. Melo. "Assisting Unknown Teammates in Unknown Tasks: Ad Hoc Teamwork under Partial Observability".
You can install the pre-compiled wheel file using pip.
pip install overcooked-ai
Note that PyPI releases are stable but infrequent. For the most up-to-date development features, build from source with pip install -e .
.
It is useful to setup a conda environment with Python 3.7 (virtualenv works too):
conda create -n overcooked_ai python=3.7
conda activate overcooked_ai
Clone the repo
git clone https://github.com/HumanCompatibleAI/overcooked_ai.git
Finally, use python setup-tools to locally install
If you just want to use the environment:
pip install -e overcooked_ai/
If you also need the DRL implementations:
pip install -e overcooked_ai[harl]
When building from source, you can verify the installation by running the Overcooked unit test suite. The following commands should all be run from the overcooked_ai
project root directory:
python testing/overcooked_test.py
To check whether the humam_aware_rl is installed correctly, you can run the following script from the src/human_aware_rl directory
$ ./run_tests.sh
This will run all tests belonging to the human_aware_rl module. You can checkout the README in the submodule for instructions of running target-specific tests. This can be initiated from any directory.
If you're thinking of using the planning code extensively, you should run the full testing suite that verifies all of the Overcooked accessory tools (this can take 5-10 mins):
python -m unittest discover -s testing/ -p "*_test.py"
overcooked_ai_py
contains:
mdp/
:
overcooked_mdp.py
: main Overcooked game logicovercooked_env.py
: environment classes built on top of the Overcooked mdplayout_generator.py
: functions to generate random layouts programmatically
agents/
:
agent.py
: location of agent classesbenchmarking.py
: sample trajectories of agents (both trained and planners) and load various models
planning/
:
planners.py
: near-optimal agent planning logicsearch.py
: A* search and shortest path logic
human_aware_rl
contains:
ppo/
:
ppo_rllib.py
: Primary module where code for training a PPO agent resides. This includes an rllib compatible wrapper onOvercookedEnv
, utilities for converting rllibPolicy
classes to OvercookedAgent
s, as well as utility functions and callbacksppo_rllib_client.py
Driver code for configuing and launching the training of an agent. More details about usage belowppo_rllib_from_params_client.py
: train one agent with PPO in Overcooked with variable-MDPsppo_rllib_test.py
Reproducibility tests for local sanity checksrun_experiments.sh
Script for training agents on 5 classical layoutstrained_example/
Pretrained model for testing purposes
rllib/
:
rllib.py
: rllib agent and training utils that utilize Overcooked APIsutils.py
: utils for the abovetests.py
: preliminary tests for the above
imitation/
:
behavior_cloning_tf2.py
: Module for training, saving, and loading a BC modelbehavior_cloning_tf2_test.py
: Contains basic reproducibility tests as well as unit tests for the various components of the bc module.
human/
:
process_data.py
script to process human data in specific formats to be used by DRL algorithmsdata_processing_utils.py
utils for the above
utils.py
: utils for the repo
overcooked_demo
contains:
server/
:
app.py
: The Flask appgame.py
: The main logic of the game. State transitions are handled by overcooked.Gridworld object embedded in the game environmentmove_agents.py
: A script that simplifies copying checkpoints to agents directory. Instruction of how to use can be found inside the file or by runningpython move_agents.py -h
up.sh
: Shell script to spin up the Docker server that hosts the game
See this Google Colab for some sample code for visualizing trajectories in python.
We have incorporated a notebook that guides users on the process of training, loading, and evaluating agents. Ideally, we would like to enable users to execute the notebook in Google Colab; however, due to Colab's default kernel being Python 3.10 and our repository being optimized for Python 3.7, some functions are presently incompatible with Colab. To provide a seamless experience, we have pre-executed all the cells in the notebook, allowing you to view the expected output when running it locally following the appropriate setup.
Overcooked_demo can also start an interactive game in the browser for visualizations. Details can be found in its README
The raw data used in training is >100 MB, which makes it inconvenient to distribute via git. The code uses pickled dataframes for training and testing, but in case one needs to original data it can be found here
If you have issues or questions, don't hesitate to contact Micah Carroll at mdc@berkeley.edu.