/gym-roboy

Pip package gym environment for deep roboy control

Primary LanguagePython

Goals

This in an implementation of an OpenAI gym environment to train controllers for Roboy. They provide standardized interface definitions for RL: https://github.com/openai/gym/tree/master/gym/envs

The goals of this repo are:

  • To provide flexibility for you to train controllers for different robots (e.g. MSJ Platfrom, Upper Body, etc.).
  • Second, it is straightforward to parallelize the environments s.t. you can scale the controller training with more compute resources.
  • Lastly, we kept it easy to implement communication to your simulation engine of choice (e.g. to MuJoCo, Gazebo, etc.).

Installation

Python3.5 is required. Either use python3 and pip3 variants, or activate a Python virtual environment. If you are using the docker container deepandreinforced/rl:latest, you do not need to install this repo again. Otherwise:

python3 -m pip install -r requirements.txt
python3 -m pip install -e .

Installing this repo as a pip package is necessary if you would like to use the environment constructor form:

env = gym.make('msj-control-v1')

Structure

  • The class RoboyEnv implements gym's gym.GoalEnv. It is the central class in this repo.

Directory simulations/

  • SimulationClient is a client interface to talk to our simulations. It provides the flexibility to change simulation engines by subclassing it.
  • RosSimulationClient is a subclass of SimulationClient that uses ROS services to step and reset the simulation. We have used so far CARDSflow as our simulation engine (not part of this repo).

Directory robots/

  • RoboyRobot is the interface for any robot. You can add new robots by subclassing from it.
  • MsjRobot is a subclass of RoboyRobot with concrete dimensions and boundaries.

Class Diagram

The RoboyEnv uses a SimulationClient to read values from the simulation. The SimulationClient depends upon a RoboyRobot to parse the numbers it receives from the simulation into a meaningful RobotState

Class Diagram

Run tests

Before running the tests, make sure you have sourced ROS2 with the ROS messages of Roboy. In the docker container deepandreinforced/rl:latest, this can be done with the command source_ROS2_ROBOY_WS.

To run the unit tests:

pytest

The repo also has integration tests. To run them, you need a CARDSflow simulation running as well as the bridge between ROS1 and ROS2. Then, to run the tests, do:

cd gym_roboy/envs/tests/
bash run_all_tests.sh