This repository provides implementations of various PDEs including hyperbolic PDEs(Burger's Equation) and parabolic PDEs (Reaction-Diffusion/Heat Equations), and 2-D Navier Stokes PDEs that are all wrapped under a gym designed for boundary control problems. These problems are particularly challenging because one has to handle nonlinearity across the entire domain with only a single value at each timestep requiring the solution of PDEs in the feedback loop. See the full paper presented at L4DC 2024 for more details.
To get started, we strongly suggest the user follows the documentation tutorial for learning in a 1D environment once installation is complete.
Installing PDE Control Gym is simple, requires few dependencies, and is set to work with most Python Environments (Installation guide can be found in docuementation as well). This installation guide assumes you are familiar with either conda or virtual env in creating and installing python modules.
-
First create a new enviornment for Python 3.10+
-
Install the dependencies. PDE ContRoL Gym requires
- Gym Version 0.28.1 (Available here)
- Numpy Version 1.26.2 (Available here)
-
To run the examples listed below or any reinforcement learning algorithm, the gym is currently setup using Stable-Baselines3 Version 2.2.1.
-
Once dependencies are installed, clone the repository using
git clone https://github.com/lukebhan/PDEControlGym.git -
Then navigate to the working folder
cd PDEControlGym -
Install the package using pip in the main folder
pip install -e . -
Enjoy your fully functional PDE Control library! For usage, see the examples in either
./PDEControlGym/examplesor in the documentation or below in the examples section of this file!
Documentation for all the environments as well as tutorials are available here
Jupyter notebooks are made in the examples folder (Hyperbolic and Parabolic subfolders) that detail all the examples for both Hyperbolic and Parabolic PDEs. Additionally, full details on numerical implementation and solvers are available in the documentation. These include tutorials on how to train your own RL controller, implemented backstepping controllers as well as plotting comparisons to analyze all of your work. For managing the training of your RL controller, we recommend you use tensorboard as outlined in the notebooks. For pretrained models, please see the downloaded files on Hugging Face.
Contributions to the gym are both welcome and encouraged. Please see the contribution guidelines. Any questions are welcome to email the author Luke at lbhan@ucsd.edu
@misc{bhan2024pde,
title={PDE Control Gym: A Benchmark for Data-Driven Boundary Control of Partial Differential Equations},
author={Luke Bhan and Yuexin Bian and Miroslav Krstic and Yuanyuan Shi},
year={2024},
eprint={2405.11401},
archivePrefix={arXiv},
primaryClass={eess.SY}
}
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
