gym-pybullet-drones

This is a minimalist refactoring of the original gym-pybullet-drones repository, designed for compatibility with gymnasium, stable-baselines3 2.0, and SITL betaflight/crazyflie-firmware.

NOTE: if you prefer to access the original codebase, presented at IROS in 2021, please git checkout [paper|master] after cloning the repo, and refer to the corresponding README.md's.

Installation

Tested on Intel x64/Ubuntu 22.04 and Apple Silicon/macOS 14.1.

git clone https://github.com/utiasDSL/gym-pybullet-drones.git
cd gym-pybullet-drones/

conda create -n drones python=3.10
conda activate drones

pip3 install --upgrade pip
pip3 install -e . # if needed, `sudo apt install build-essential` to install `gcc` and build `pybullet`

Use

PID control examples

cd gym_pybullet_drones/examples/
python3 pid.py # position and velocity reference
python3 pid_velocity.py # desired velocity reference

Downwash effect example

cd gym_pybullet_drones/examples/
python3 downwash.py

Reinforcement learning examples (SB3's PPO)

cd gym_pybullet_drones/examples/
python learn.py # task: single drone hover at z == 1.0
python learn.py --multiagent true # task: 2-drone hover at z == 1.2 and 0.7

utiasDSL `pycffirmware` Python Bindings example (multiplatform, single-drone)

Install pycffirmware for Ubuntu, macOS, or Windows

cd gym_pybullet_drones/examples/
python3 cff-dsl.py

Betaflight SITL example (Ubuntu only)

git clone https://github.com/betaflight/betaflight # use the `master` branch at the time of writing (future release 4.5)
cd betaflight/ 
make arm_sdk_install # if needed, `apt install curl``
make TARGET=SITL # comment out line: https://github.com/betaflight/betaflight/blob/master/src/main/main.c#L52
cp ~/gym-pybullet-drones/gym_pybullet_drones/assets/eeprom.bin ~/betaflight/ # assuming both gym-pybullet-drones/ and betaflight/ were cloned in ~/
betaflight/obj/main/betaflight_SITL.elf

In another terminal, run the example

conda activate drones
cd gym_pybullet_drones/examples/
python3 beta.py --num_drones 1 # check the steps in the file's docstrings to use multiple drones

Citation

If you wish, please cite our IROS 2021 paper (and original codebase) as

@INPROCEEDINGS{panerati2021learning,
      title={Learning to Fly---a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control}, 
      author={Jacopo Panerati and Hehui Zheng and SiQi Zhou and James Xu and Amanda Prorok and Angela P. Schoellig},
      booktitle={2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
      year={2021},
      volume={},
      number={},
      pages={7512-7519},
      doi={10.1109/IROS51168.2021.9635857}
}

References

Carlos Luis and Jeroome Le Ny (2016) Design of a Trajectory Tracking Controller for a Nanoquadcopter
Nathan Michael, Daniel Mellinger, Quentin Lindsey, Vijay Kumar (2010) The GRASP Multiple Micro UAV Testbed
Benoit Landry (2014) Planning and Control for Quadrotor Flight through Cluttered Environments
Julian Forster (2015) System Identification of the Crazyflie 2.0 Nano Quadrocopter
Antonin Raffin, Ashley Hill, Maximilian Ernestus, Adam Gleave, Anssi Kanervisto, and Noah Dormann (2019) Stable Baselines3
Guanya Shi, Xichen Shi, Michael O’Connell, Rose Yu, Kamyar Azizzadenesheli, Animashree Anandkumar, Yisong Yue, and Soon-Jo Chung (2019) Neural Lander: Stable Drone Landing Control Using Learned Dynamics
C. Karen Liu and Dan Negrut (2020) The Role of Physics-Based Simulators in Robotics
Yunlong Song, Selim Naji, Elia Kaufmann, Antonio Loquercio, and Davide Scaramuzza (2020) Flightmare: A Flexible Quadrotor Simulator

Core Team WIP

Multi-drone crazyflie-firmware SITL support (@spencerteetaert, @JacopoPan)
Use SITL services with steppable simulation (@JacopoPan)

Desired Contributions/PRs

Add motor delay, advanced ESC modeling by implementing a buffer in BaseAviary._dynamics()
Replace rpy with quaternions (and ang_vel with body rates) by editing BaseAviary._updateAndStoreKinematicInformation(), BaseAviary._getDroneStateVector(), and the .computeObs() methods of relevant subclasses

Troubleshooting

On Ubuntu, with an NVIDIA card, if you receive a "Failed to create and OpenGL context" message, launch nvidia-settings and under "PRIME Profiles" select "NVIDIA (Performance Mode)", reboot and try again.

Run all tests from the top folder with

pytest tests/

University of Toronto's Dynamic Systems Lab / Vector Institute / University of Cambridge's Prorok Lab

perevalovds/gym-pybullet-drones