An gym environment for acrobot in VREP, along with a sample script for training the acrobot agent using DDPG (deep deterministic policy gradient). Using a similar approach a VREP-based reinforcement learning setup can be created for more complicated control problems, or for other RL algorithms.
-
acrobotVREP - Python package for acrobotVREP (contains acrobotVrep.py
1.1 acrobotVREP/env/acrobotVrep.py - Gym class for acrobotVREP
1.2 acrobotVREP/env/transformations.py - A matrix library by Christoph Gohlke http://www.lfd.uci.edu/~gohlke/
-
vrep_scenes - VREP scene file
-
acrobot_train.py - Script for training or testing the acrobot agent
- Python 2.7, numpy (not tested on Python 3)
- OpenAI Gym (https://gym.openai.com/)
- Pypot (https://poppy-project.github.io/pypot/) - provides an easy way of interacting with VREP
- Keras (https://keras.io/)
- Keras-rl (https://github.com/keras-rl/keras-rl)
- VREP simulator (http://www.coppeliarobotics.com/)
-
Set the $HOME location to where the repository can be stored:
### Change the path as required export HOME=/set/as/needed
-
Clone the repository (or copy it from the disk)
cd $HOME mkdir -p $HOME/computing/repositories/ cd $HOME/computing/repositories/ git clone https://github.com/sayantanauddy/acrobotVREP.git
-
Download VREP
cd $HOME mkdir $HOME/computing/simulators/ cd $HOME/computing/simulators/ # Download wget http://coppeliarobotics.com/files/V-REP_PRO_EDU_V3_4_0_Linux.tar.gz # Extract tar -xvf V-REP_PRO_EDU_V3_4_0_Linux.tar.gz
-
Create the virtual environment
cd $HOME virtualenv --system-site-packages $HOME/acrobot_virtualenv # Activate the virtual environment source $HOME/acrobot_virtualenv/bin/activate
-
Add the code location to PYTHONPATH
export PYTHONPATH=$PYTHONPATH:$HOME/computing/repositories/acrobotVREP
-
Install the dependencies
# numpy should also be installed pip install pypot pip install tensorflow pip install tensorflow==1.5 pip install keras pip install keras-rl pip install gym pip install h5py
-
Start VREP in a separate terminal
cd $HOME/computing/simulators/V-REP_PRO_EDU_V3_4_0_Linux ./start_vrep.sh
-
Run acrobot_train.py. Same script can be used for testing also (check comments in the script). VREP shows a prompt when run for the first time. Tick the checkbox in the prompt and close it. This needs to be done 3 times.
cd $HOME/computing/repositories/acrobotVREP python acrobot_train.py