/MARL_CAVs

MARL for autonomous vehicles.

Primary LanguagePython

Update

Add multi-agent parking environment

output_example
Parking environment

You can try this out with:

import highway_env
import gym

env = gym.make("parking-multi-agent-v0")

for _ in range(100):

    obs, reward, terminal, info = env.step(env.action_space.sample())
    env.render()

Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic

An on-policy MARL algorithm for highway on-ramp merging problem, which features parameter sharing, action masking, local reward design and a priority-based safety supervisor.

Algorithms

All the MARL algorithms are extended from the single-agent RL with parameter sharing among agents.

  • MAA2C: TBD.
  • MAPPO.
  • MAACKTR.
  • MADQN: Does not work well.
  • MASAC: TBD.

Installation

  • create an python virtual environment: conda create -n marl_cav python=3.6
  • active the virtul environment: conda activate marl_cav
  • install pytorch (torch>=1.2.0): pip install torch===1.7.0 torchvision===0.8.1 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
  • install the requirements: pip install -r requirements.txt

output_example
Fig.1 Illustration of the considered on-ramp merging traffic scenario. CAVs (blue) and HDVs (green) coexist on both ramp and through lanes.

Demo

see the site

Usage

To run the code, just run it via python run_xxx.py. The config files contain the parameters for the MARL policies.

Training curves

output_example
Fig.2 Performance comparison between the proposed method and 3 state-of-the-art MARL algorithms.

Cite

@misc{chen2021deep,
      title={Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic}, 
      author={Dong Chen and Zhaojian Li and Yongqiang Wang and Longsheng Jiang and Yue Wang},
      year={2021},
      eprint={2105.05701},
      archivePrefix={arXiv},
      primaryClass={eess.SY}
}

Reference