Leibniz Gym

This repository provides IsaacGym environment for the TriFinger robot. However, the development work can be used to base other RL environments in IsaacGym as well.

The project currently uses RL-Games for training agents. More support will follow based on demand.

For list of contributors, check: CONTRIBUTORS. This code is released under LICENSE.

Installation

Details regarding installation of IsaacGym can be found here. We currently support the Preview Release 2 version of IsaacGym.

Pre-requisites

The code has been tested on Ubuntu 20.04 with Python 3.7. The minimum recommended NVIDIA driver version for Linux is 460.32 (dictated by support of IsaacGym).

It uses Anaconda to create virtual environments. To install Anaconda, follow instructions here.

Install IsaacGym in a new conda environment

Run the IsaacGym installation script provided in the downloaded IsaacGym zip):

cd PATH/TO/isaacgym
bash create_conda_env_rlgpu.sh

This will create a new conda env called rlgpu, which can be activated by running:

conda activate rlgpu

Install Leibnizgym

To install leibnizgym package and all its dependencies, run:

cd PATH/TO/leibnizgym
pip install -e .

Install RL toolbox (optional)

Currently, we use RL-Games for reinforcement learning.

It should be downloaded automatically with the previous step.

However, if you want to try with a different branch, you can manually install it locally instead:

# somewhere outside trifinger-manip
git clone https://github.com/Denys88/rl_games.git
pip install -e .

Running

Training

We use Hydra to keep configuration of runs simple. You can view the main arguments to the scripts by looking in the file resources/config/config.yaml.

You can also set the configuration parameters from terminal by doing {config_variable_name}={value}. The main ones to be aware of are:

gym (string): environment name to use. The current options are the different difficulty levels of trifinger: trifinger_difficulty_{1,2,3,4}
num_envs (int): number of environment instances to run.
headless (bool): whether to run the simulator with/without GUI.

To train an agent on environment difficulty 2, 8192 environment instances and headless mode, run:

python scripts/rlg_hydra.py gym=trifinger_difficulty_2 args.num_envs=8192 args.headless=True

Inference and Loading Checkpoints

Trained checkpoints from RL-Games are saved within the nn/ directory within the relevant output directory (output/<date>/time by default)

To perform inference and see the results, you can run:

python scripts/rlg_hydra.py args.num_envs=<INT> args.play=True args.checkpoint=/path/to/checkpoint.pth args.headless=False

where:

play tells it not to train
checkpoint is the path to load the checkpoint.

If you want to start training from a trained checkpoint, then just don't specify play (or set it to False). Usually you'll only want a few environments (say, 256) to get smooth rendering performance during inference.

Citing

@article{allshire2021transferring,
  title={Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger},
  author={Allshire, Arthur and Mittal, Mayank and Lodaya, Varun and Makoviychuk, Viktor and Makoviichuk, Denys and Widmaier, Felix and W{\"u}thrich, Manuel and Bauer, Stefan and Handa, Ankur and Garg, Animesh},
  journal={arXiv preprint arXiv:2108.09779},
  year={2021}
}