Incremental Reinforcement Learning in Continuous Spaces

This repo contains code accompaning the paper: Zhi Wang, Han-Xiong Li, and Chunlin Chen, "Incremental reinforcement learning in continuous spaces via policy relaxation and importance weighting", IEEE Transactions on Neural Networks and Learning Systems, 2019. It contains code for running the incremental learning domain tasks, including 2D navigation, Reacher, Swimmer, Hopper, and HalfCheetah domains.

Dependencies

This code requires the following:

python 3.5+
pytorch 0.4+
gym
MuJoCo license

Data

For the 2D navigation domain, data is generated from myrllib/envs/navigation.py
For the Reacher/Swimmer/Hopper/HalfCheetah Mujoco domains, the modified Mujoco enviornments are in myrllib/envs/mujoco/*

Usage

For example, to run the code in the 2D Navigation domain with type I dynamic environment, just run the bash script navigation_v1.sh, also see the usage instructions in the script and main.py
When getting the results in output/*/*.npy files, plot the results using data_process.py. For example, the results for navigation_v3.sh and swimmer.sh are as follows:

navigation_v3	swimmer

Also, the results for other scripts are shown in exp/*

Remark

All the scripts deal with one demo trial in all investigated domains. To obtain the results as shown in the paper, implement many trials with random environment changes.

Contact