IsaacGym Bimanual Franka task

This repo is based on Nvidia's repo IsaacGymEnvs, edited by me for the ICIRA conference and RAS conference experiment.

In this work, we propose new training methods for multi-agent RL. And we will provide Isaac Gym's users a test environment to help you monitor your training process. Moreover, some multi-agent APIs will be provided for multi-agent training in Isaac Gym.

A part of our work will be published at the ICIRA conference; the other part is still working and expected to be accepted by RAS.

IsaacGym Bimanual Franka task
- Handbook about our work
- Work achievement
  - About ICIRA
  - About RAS

Handbook about our work

This is a handbook about our work's difference with Nvidia's work.

We provide a test platform where you can view the rewards, obs, or any information you want to know during the training process. Moreover, it can record offline data in Simulate environment to apply the offline algorithm without a real environment.

It provides you more choices to train your tasks at a low cost.
We will provide a multi-agent train API with multi-PPO, multi-sac algorithms to help you train your multi-agent tasks.

Installation

Create the conda environment

conda env create -f config/environment.yml

Install pytorch with the correct CUDA version from, 1.12.1 was tested.
Install IssacGym
Install this repo
```
pip3 install -e .
```
sudo cp /home/ubuntu/anaconda3/envs/pytorch/lib/libpython3.8.so.1.0 /usr/lib

Install Ninja

wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
sudo unzip ninja-linux.zip -d /usr/local/bin/
sudo update-alternatives --install /usr/bin/ninja ninja /usr/local/bin/ninja 1 --force

Pack this repo to replace the official isaacgymenvs package
python3 setup.py bdist_wheel 
python3 setup.py sdist
pip3 install -e .

Test environment during the training

Place test_new.py under the same folder with train.py, should be /IsaacGymEnvs/isaacgymenvs
Check the test config file in test_config.ini
Running in the environment
```
python test.py
```
Save pose data under folder ./test_save, hdf5 under ./test_save/hdf5

Test environment's keyboard shortcut

Keyboard shortcut	Description
`R`	Reset the environment. Need to disable pose override in the viewer first.
`T`	Print the obs/reward once in terminal.
`P`	Turn on/off the print info.
`C`	Switch the view angle
`S`	Save the pose status in file.
`0`	Enable/disable target tracking
`N`	Force jump to next stage in auto tracking
`D`	Enable/disable debug info
`V`	Check task stage in certain obs
`9`	Pause tracking
`L`	Override Franka DOFs
`UP, DOWN, LEFT, RIGHT`	Drive franka position manually in `xz` plane
`[`, `]`	Drive franka position manually in `y` axis
`LEFT_SHIFT`	Switch driving between left/right Franka

Work achievement

We try to show you our efforts from two papers; the ICIRA one is already accepted as soon as the RAS one is still working!

About ICIRA

Title: 'Mixline: A Hybrid Reinforcement Learning Framework for Long-horizon Bimanual Coffee Stirring Task'

The camera-ready version:

https://mycuhk-my.sharepoint.com/:b:/g/personal/1155169521_link_cuhk_edu_hk/ETAQRHX-UcRJr2E4gKzROZgBYWgVX_5-VrLBOjP8WaIq0w?e=l3X3hg

The powerpoint about our ICIRA work:

https://mycuhk-my.sharepoint.com/:p:/g/personal/1155169521_link_cuhk_edu_hk/EaD-nU8Dc2hHreP4dB2QQ-oB5EnMBSCWO4AFzSCv7wUfpQ?e=D1nHnh

Mixline diagram:

About RAS

The paper for RAS is still working.

ZainZh/IsaacGymEnvs_ICIRA