/MPLight

Primary LanguagePython

TrafficLightRL

This is a new version of a reinforcement learning pipeline for intelligent traffic control. Usage and more information can be found below.

Usage

Start an experiment:

python -O run_batch.py

Here, -O option cannot be omitted unless debug is necessary. In the file run_batch.py, the name of memo can be changed. There are four modules executed sequentially:

  • runexp.py

    Run the pipeline under different traffic flows. Specific traffic flow files as well as basic configuration can be assigned in this file. For details about config, please turn to config.py.

  • testexp.py

    With models trained after different rounds, we can test them using this module. run_cnts can be adjusted.

  • summary_detail.py

    Calculate durations with our models as control agent for traffic light in the train and test mode and write results to train_results.csv and test_results.csv respecitvely. At the same time, add summaries of this experiment setting to total_train_results.csv and total_test_results.csv.

  • summary_plot.py

    Plot figures of Round V.S Duration with obtained results.

For most cases, you might only modify memo in run_batch.py, traffic files and config parameters in runexp.py.

Agent

  • agent.py

    A abstract class of different agents.

  • ./baseline/fixedtime_agent.py

    An agent to implement fixedtime control on traffic lights.

  • network_agent.py

    A abstract class of neural network based agents. All methods are defined in this file but build_network(), which means only build_network() is necessary in specific network agents.

  • ./baseline/deeplight_agent.py

    A DQN agent with phase selecor.

  • simple_dqn_agent.py

    A simple DQN agent

Others

More details about this project are demonstrated in this part.

  • config.py

    The whole configuration of this project. Note that some parameters will be replaced in runexp.py while others can only be changed in this file, please be very careful!!!

  • pipeline.py

    The whole pipeline is implemented in this module:

    Start a SUMO environment, run a simulation for certain time(one round), construct samples from raw log data, update the model and model pooling.

  • generator.py

    A generator to load a model, start a SUMO enviroment, conduct a simulation and log the results.

  • sumo_env.py

    Define a SUMO environment to interact with SUMO and obtain needed data like features.

  • construct_sample.py

  • Construct training samples from original data. Select desired state features in the config and compute the corrsponding average/instant reward with specific measure time.

  • updater.py

    Define a class of updater for model updating.

baseline

to run some baseline algorithms (formula, fixedtime, lit (deeplight)):

  • python -O run_baseline_batch.py