Haishanliu/sumo-rl

A simple interface to instantiate Reinforcement Learning environments with SUMO for Traffic Signal Control. Compatible with Gym Env from OpenAI and MultiAgentEnv from RLlib.

SUMO-RL

SUMO-RL provides a simple interface to instantiate Reinforcement Learning environments with SUMO for Traffic Signal Control.

The main class SumoEnvironment inherits MultiAgentEnv from RLlib.
If instantiated with parameter 'single-agent=True', it behaves like a regular Gym Env from OpenAI.
TrafficSignal is responsible for retrieving information and actuating on traffic lights using TraCI API.

Goals of this repository:

• Provide a simple interface to work with Reinforcement Learning for Traffic Signal Control using SUMO
• Support Multiagent RL
• Compatibility with gym.Env and popular RL libraries such as stable-baselines3 and RLlib
• Easy customisation: state and reward definitions are easily modifiable

Install

Install SUMO latest version:

sudo add-apt-repository ppa:sumo/stable
sudo apt-get update
sudo apt-get install sumo sumo-tools sumo-doc 

Don't forget to set SUMO_HOME variable (default sumo installation path is /usr/share/sumo)

echo 'export SUMO_HOME="/usr/share/sumo"' >> ~/.bashrc
source ~/.bashrc

Important: for a huge performance boost (~8x) with Libsumo, you can declare the variable:

export LIBSUMO_AS_TRACI=1

Notice that you will not be able to run with sumo-gui if this is active (more details).

Install SUMO-RL

Stable release version is available through pip

pip install sumo-rl

Alternatively you can install using the latest (unreleased) version

git clone https://github.com/LucasAlegre/sumo-rl
cd sumo-rl
pip install -e .

Examples

Check experiments to see how to instantiate a SumoEnvironment and use it with your RL algorithm.

Q-learning in a one-way single intersection:

python3 experiments/ql_single-intersection.py 

RLlib A3C multiagent in a 4x4 grid:

python3 experiments/a3c_4x4grid.py

stable-baselines3 DQN in a 2-way single intersection:

python3 experiments/dqn_2way-single-intersection.py

Plotting results:

python3 outputs/plot.py -f outputs/2way-single-intersection/a3c 

Citation

If you use this repository in your research, please cite:

@misc{sumorl,
    author = {Lucas N. Alegre},
    title = {{SUMO-RL}},
    year = {2019},
    publisher = {GitHub},
    journal = {GitHub repository},
    howpublished = {\url{https://github.com/LucasAlegre/sumo-rl}},
}