/Diffusion_RL

This repo has the code and suplementary materials of our 2024 RAL submission.

Primary LanguagePython

Diffusion_RL

This repo has the code and suplementary materials of our 2024 RAL submission "Diffusion-Reinforcement Learning Hierarchical Motion Planning in Adversarial Multi-agent Games".

Code Introduction

  • Our code is mainly in the folder red_rl_diffusion.
  • You can first collect the RRT* dataset with the file collect_demonstrations_balance_game.py in the folder datasets.
  • Then the diffusion model can be trained with the function red_diffusion_train in the file red_diffusion_main.py.
  • Next, our diffusion-RL model and DDPG, SAC baselines can be trained from red_rl_main.py file with the functions in it.
  • You can use the file red_rl_test.py to generate the benchmarking results including learning based methods and heuristics.
  • The results can be visualized with the file red_rl_analysis.py. It will print out the benchmarking results and generate boxplots.
  • Remember all the log files and checkpoints of the model you trained will be in the folder logs.

Conda Environment

  • Please check the file environment.yml for the package versions we are using.

Video

Citations

  • We adapt our diffusion model training code from Diffuser with the following citation:
@inproceedings{janner2022diffuser,
  title = {Planning with Diffusion for Flexible Behavior Synthesis},
  author = {Michael Janner and Yilun Du and Joshua B. Tenenbaum and Sergey Levine},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
}