/project1-boptest-gym

Primary LanguagePythonOtherNOASSERTION

BOPTEST-Gym

BOPTESTS-Gym is the OpenAI-Gym environment for the BOPTEST framework. This repository accommodates the BOPTEST API to the OpenAI-Gym convention in order to facilitate the implementation, assessment and benchmarking of reinforcement learning (RL) algorithms for their application in building energy management. RL algorithms from the Stable-Baselines 3 repository are used to exemplify and test this framework.

The environment is described in this paper.

Structure

  • boptestGymEnv.py contains the core functionality of this Gym environment.
  • environment.yml contains the dependencies required to run this software.
  • /examples contains prototype code for the interaction of RL algorithms with an emulator building model from BOPTEST.
  • /testing contains code for unit testing of this software.

Quick-Start

  1. Create a conda environment from the environment.yml file provided (instructions here).
  2. Run a BOPTEST case with the building emulator model to be controlled (instructions here).
  3. Develop and test your own RL algorithms. See example below using the Bestest hydronic case with a heat-pump and the A2C algorithm from Stable-Baselines:
from boptestGymEnv import BoptestGymEnv, NormalizedActionWrapper, NormalizedObservationWrapper
from stable_baselines3 import A2C
from examples.test_and_plot import test_agent

# BOPTEST case address
url = 'http://127.0.0.1:5000'

# Instantite environment
env = BoptestGymEnv(url                   = url,
                    actions               = ['oveHeaPumY_u'],
                    observations          = {'reaTZon_y':(280.,310.)}, 
                    random_start_time     = True,
                    max_episode_length    = 24*3600,
                    warmup_period         = 24*3600,
                    step_period           = 900)

# Add wrappers to normalize state and action spaces (Optional)
env = NormalizedObservationWrapper(env)
env = NormalizedActionWrapper(env)  

# Instantiate and train an RL algorithm
model = A2C('MlpPolicy', env)
model.learn(total_timesteps=int(1e5))

# Test trained agent
observations, actions, rewards, kpis = test_agent(env, model, 
                                                  start_time=0, 
                                                  episode_length=14*24*3600,
                                                  warmup_period=24*3600,
                                                  plot=True)

Citing the project

Please use the following reference if you used this repository for your research.

@inproceedings{boptestgym2021,
	author = {Javier Arroyo and Carlo Manna and Fred Spiessens and Lieve Helsen},
	title = {{An OpenAI-Gym environment for the Building Optimization Testing (BOPTEST) framework}},
	year = {2021},
	month = {September},
	booktitle = {Proceedings of the 17th IBPSA Conference},
	address = {Bruges, Belgium},
}