Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity

Robotics Institute, Carnegie Mellon University

Conference on Robot Learning (CoRL) 2022 (Oral)

In this paper, we build a system based on reinforcement learning that shows emergent extrinsic dexterity behavior with a simple gripper for the "Occluded Grasping" task. This repository contains the code for the simulation environment of the Occluded Grasping task and RL training and rollouts. The code for the real robot can be found in a separate repository.

This repository is built on top of robosuite-benchmark. The simulation environment is based on robosuite and the RL training related code is based on rlkit. As an overview of this repository, ungraspable/robosuite_env defines the Occluded Grasping task. ungraspable/rlkit_utils defines helper functions to be used with rlkit.

Please feel free to contact us if you have any questions on the code or anything else related to our paper!

Installation

Clone the current repository:

$ git clone --recursive https://github.com/Wenxuan-Zhou/ungraspable.git
$ cd ungraspable

Create a conda environment with required packages. IMPORTANT: We require the exact version of robosuite and rlkit included in this directory (included in the following yml file).

conda env create -f env.yml
source activate ungraspable

Use viskit to visualize training log files. Do not install it in the above conda environment because there are compatibility issues.

Usage

Training

python ungraspable/train.py --ExpID 0000

The results will be saved under "./results" by default. During training, you can visualize current logged runs using viskit.

To train the policy with a multi-grasp curriculum:

python ungraspable/train.py --adr_mode 0001_ADR_MultiGrasp --ExpID 0001 --goal_range use_threshold

"--adr_mode" specified an ADR configuration file under ungraspable/rlkit_utils/adr_config. Similarly, to train the policy with Automatic Domain Randomization over physical parameters:

python ungraspable/train.py --adr_mode 0002_ADR_physics --ExpID 0002

We include the results of the above training commands in result/examples, including the model and the training logs. You may visualize the training curves of these examples using viskit:

python your_viskit_folder/viskit/frontend.py ungraspable/results/examples

Visualizing Rollouts

To visualize a trained policy with onscreen mujoco renderer:

python ungraspable/rollout.py --load_dir results/examples/Exp0000_OccludedGraspingSimEnv_tmp-0 --camera sideview --grasp_and_lift

Feel free to try out other checkpoints in the result/examples folder.

Citation

If you find this repository useful, please cite our paper:

@inproceedings{zhou2022ungraspable,
  title={Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity},
  author={Zhou, Wenxuan and Held, David},
  booktitle={Conference on Robot Learning (CoRL)},
  year={2022}
}