/elevator-scheduling

Simulation environment for elevator scheduling.

Primary LanguagePython

About

An elevator scheduling simulation environment to be used for the development of (reinforcement) learning algorithms in Python.

Includes example agent implementations for:

  • Tabular Q Learning agent (agent/tabular_q_learning.py)
  • DQN agent (dqn_controller.py)
  • Categorical DQN agent (cdqn_controller.py)
  • Evolution Strategy agent (es_controller.py)
  • REINFORCE (reinforce_controller.py)
  • Heuristic benchmarks
    • FCFS Round Robin
    • Random Policy
    • (Static) Zoning

Call generating policies implemented include:

  • Pure up-peak traffic
  • Pure down-peak traffic
  • Pure interfloor traffic
  • Mixed traffic (e.g. observed at lunch time in an office building)

Architecture

  • Agent
    • Given a building state representation, agent must respond with action to be executed for building (e.g. let elevator 1 ascend, elevator 2 stop, etc.)
    • Abstract class Agent (agent/agent.py) describes interface controllers can program against
  • Building
    • Acts as the environment the agent operates in
    • Abstract class Building (building/building.py) describes interface controllers can program against
    • building/tf_building.py wraps Building environment as a tf-agents PyEnvironment for use in tf-agents agents
  • Caller
    • Implements call generating policy of building
    • Abstract class Caller (caller/caller.py) describes interface Building can program against
  • Controller
    • Ties together all moving parts of the system (i.e. agent, building (environment), caller)
    • For reinforcement learning agents, the controller implements the staple sample-act-sample-update loop
  • settings.py
    • Contains global constants of the simulation environment and the learning agents (e.g. number of floors, number of elevators, etc.)