mcppoElegantRLforCarla

There is the implementation of multi-constraint proximal policy optimization (MCPPO), which is modified based on a DRL framework -- ElegantRL(小雅). The carla environment is modified based on gym-carla. Please refer to our paper for details.

Get started

OS and Carla

Our experimental environment is based on Ubuntu18.04 and Carla0.9.11 Before training in Carla, you need

install Carla, the tutorial: https://carla.readthedocs.io/en/latest/start_quickstart/
install Docker, the tutorial: https://docs.docker.com/engine/install/ubuntu/
install Nvidia-docker2 (for running env by GPU), the tutorial: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker
start carla server, the tutorial: https://carla.readthedocs.io/en/latest/build_docker/#nvidia-docker2 or start with python ./gym_carla_feature/start_env/demo/startServer.py

Python environment

Please see requirements.txt

pip install -r requirements.txt

You can run PID control to test

python gym_carla_feature/start_env/demo/rule_demo.py

to test if carla is installed correctly.

Training

After runing carla server, you can start a training with a quickstart shell

bash ./start/carla_veh_control_run.sh

, which includes PPO, MCPPO, SAC, D3QN. Or start with

python gym_carla_feature/start_env/demo/veh_control.py

You also use ray_elegantrl to try other envs such as

python ray_elegantrl/demo.py

Evaluation

The tensorboard file and model file will be saved in ./veh_control_logs

Test

The policy our trained is in ./veh_control_logs/veh_control/town07-V1.

Run PID drive policy by

python gym_carla_feature/start_env/demo/rule_demo.py

Run DRL drive policy by

python gym_carla_feature/start_env/demo/eval_policy_carla.py

Citation

@inproceedings{zou2022mcppo,
  author    = {Guangyuan Zou and
               Ying He and
               F. Richard Yu and
               Longquan Chen and
               Weike Pan and
               Zhong Ming},
  title     = {Multi-Constraint Deep Reinforcement Learning for Smooth Action Control},
  booktitle = {Proceedings of the Thirty-First International Joint Conference on
               Artificial Intelligence, {IJCAI} 2022, Vienna, Austria, 23-29 July
               2022},
  pages     = {3802--3808},
  publisher = {ijcai.org},
  year      = {2022},
  url       = {https://doi.org/10.24963/ijcai.2022/528}
}

Thanks

Thanks for 小雅 and Yinyan Zeng, the author. I learned a lot from this repository.

OrionZou/mcppoElegantRLforCarla