/neuriss

Official Inplementation of L4DC paper: Compositional Neural Certificates for Networked Dynamical Systems

Primary LanguagePythonMIT LicenseMIT

NeurISS

Conference Arxiv Conference

Official implementation of the L4DC 2023 paper: S Zhang, Y Xiu, G Qu, C Fan: "Compositional Neural Certificates for Networked Dynamical Systems".

Dependencies

We recommend to use CONDA to install the requirements:

conda create -n neuriss python=3.10
conda activate neuriss
pip install -r requirements.txt

Install

Install NeurISS:

pip install -e .

Run

Environments

We provide 3 environments including Microgrid-IEEE5, PlatoonSin5, and PlanarDroneSin2x2. One can also create other environments by changing the number of agents. For example, one can use PlatoonSin10, PlanarDroneSin3x2. However, for the microgrid environment, we only provide Microgrid-IEEE4 and Microgrid-IEEE5 because this environment depends on the data provided in [1].

Hyper-parameters

To reproduce the results shown in our paper, one can refer to hyperparams.yaml.

Train NeurISS

Use the following command to train NeurISS:

python train_neuriss.py --env Microgrid-IEEE5 --n-iter 10000 --init-iter 0

Training options:

  • --env: name of the environment
  • --n-iter: number of training iterations
  • --init-iter: number of initializing iterations
  • --batch-size: batch size of training data
  • --adjust-weight: decrease the weight of controller loss after some iteration
  • --u-nominal: path to the pre-defined nominal controller
  • --gpus: index of the training gpu
  • --no-cuda: disable cuda
  • --log-path: path to save training logs
  • --seed: random seed

Train RL

Use the following command to train RL algorithms:

python train_rl.py --env Microgrid-IEEE5 --algo ppo --steps 5000000

We provide the implementation of 3 RL algorithms including PPO [2], LYPPO [3], and MAPPO [4].

Training options:

  • --env: name of the environment
  • --algo: name of the algorithm
  • --steps: number of training steps
  • --gpus: index of the training gpu
  • --seed: random seed
  • --log-path: path to save training logs
  • --no-cuda: disable cuda

Train NCLF

Use the following command to train NCLF [5]:

python train_clf.py --env Microgrid-IEEE5 --n-iter 10000

Training options:

  • --env: name of the environment
  • --n-iter: number of training iterations
  • --gpus: index of the training gpu
  • --seed: random seed
  • --no-cuda: disable cuda
  • --log-path: path to save training logs
  • --batch-size: batch size of training data

Evaluation

Use the following command to test the agents:

python test.py --path <path-to-log> --epi 20

Use the following command to test the nominal controller

python test.py --env Microgrid-IEEE5 --epi 20

Options:

  • --path: path to the log folder (start with seed)
  • --epi: number of episode
  • --env: name of the environment (no need to specify if you want to test on the training environment)
  • --iter: the iteration of the controller (no need to specifiy if you want to test the best controller)
  • --no-cuda: disable cuda
  • --no-video: do not make video (for faster test)

To test in an environment with different number of agents, use --env to specify the environment you want to test on.

Tracking Training Process

We use TensorBoard to track the training process. Use the following line to see how everything goes on during the training:

tensorboard --logdir='<path-to-log>'

Pre-trained Models

We provide the pre-trained models in the folder ./pretrained.

Citation

@inproceedings{zhang2023neuriss,
      title={Compositional Neural Certificates for Networked Dynamical Systems},
      author={Songyuan Zhang and Yumeng Xiu and Guannan Qu and Chuchu Fan},
      booktitle={5th Annual Learning for Dynamics {\&} Control Conference},
      year={2023},
}

References

[1] Tong Huang, Sicun Gao, and Le Xie. A neural lyapunov approach to transient stability assessment of power electronics-interfaced networked microgrids. IEEE Transactions on Smart Grid, 13(1): 106–118, 2021.

[2] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.

[3] Ya-Chien Chang and Sicun Gao. Stabilizing neural control using self-learned almost lyapunov critics. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 1803–1809. IEEE, 2021.

[4] Chao Yu, Akash Velu, Eugene Vinitsky, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effectiveness of ppo in cooperative multi-agent games. arXiv preprint arXiv:2103.01955, 2021.