SUMO-gym

OpenAI-gym like toolkit for developing and comparing reinforcement learning algorithms on SUMO.

This repo provides the source codes for "SMART-eFlo: An Integrated SUMO-Gym Framework for Multi-Agent Reinforcement Learning in Electric Fleet Management Problem". Please cite it if you find it helpful.

@INPROCEEDINGS{9922047,
  author={Liu, Shuo and Wang, Yunhao and Chen, Xu and Fu, Yongjie and Di, Xuan},
  booktitle={2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC)}, 
  title={SMART-eFlo: An Integrated SUMO-Gym Framework for Multi-Agent Reinforcement Learning in Electric Fleet Management Problem}, 
  year={2022},
  volume={},
  number={},
  pages={3026-3031},
  doi={10.1109/ITSC55140.2022.9922047}}

Installation

Install SUMO, SUMO GUI and XQuartz according to official guide.

$ python3 -m venv .env
$ source .env/bin/activate
(.env)$ pip install -r requirements.txt
(.env)$ pip install sumo-gym
(.env)$ export SUMO_HOME=<your_path_to>/sumo SUMO_GUI_PATH=<your_path_to>/sumo-gui # and copy the paths to ~/.bashrc

The installation is successful so far, then you can try the examples in the tutorials, for example:

(.env)$ python3 tutorials/fmp-jumbo.py --render 0

Features

SUMO-gym aims to build an interface between SUMO and Reinforcement Learning. With this toolkit, you will be able to convert the data generated from SUMO simulator into RL training setting like OpenAI-gym.

Remarkable features include:

OpenAI-gym RL training environment based on SUMO.

import gym
from sumo_gym.envs.fmp import FMP

env = gym.make(
    "FMP-v0", mode, n_vertex, n_edge, n_vehicle, 
    n_electric_vehicles, n_charging_station, 
    vertices, demand, edges, 
    electric_vehicles, departures, charging_stations,
)
for _ in range(n_episode):
    obs = env.reset()
    for t in range(n_timestamp):
        action = env.action_space.sample()
        obs, reward, done, info = env.step(action)
        if done:
            break
env.close()