A Gym wrapper for the Franka Emika Panda robot based on FrankaPy. This repository includes the code for the real robot experiments in Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity. You may find the simulation environment used for Sim2Real transfer in this repository.
The gym environment of the franka arm is defined in frankapy_env/franka_env.py. ROS point cloud processing code and object pose estimation based on Iterative closest point (ICP) can be found in frankapy_env/pointcloud.py. If you are interested in the details of the Occluded Grasping task in our paper, please checkout frankapy_env/occluded_grasping_env.py.
Please feel free to contact us if you have any questions on the code or anything else related to our paper!
- Robot Setup: Install frankapy and franka-interface by following the instructions here.
- Camera Setup (Azure Kinect):
- Install Azure SDK:
- Useful installation notes
- Official instructions
- Before going to the next step, verify the camera reading in "k4aviewer".
- Install Azure Kinect ROS Driver.
- Run camera calibration using easy_hand_eye.
- Replace the yaml file in this repository: easy_handeye_eye_on_base.yaml
- Install Azure SDK:
- Install this repository.
pip install -r requirements.txt - (Optional) If you want to try the example code of policy rollout, you also need to install rlkit and pytorch.
- You only need to
pip install git+https://github.com/rail-berkeley/rlkit.gitfor rlkit without installing additional dependencies if you just want to try out the rollout code in this repository.
- You only need to
To try out the FrankaPy Gym Environment:
python scripts/test_env.py
To rollout with a policy:
python scripts/rollout_policy.py models/example.pkl
If you find this repository useful, please cite our paper:
@inproceedings{zhou2022ungraspable,
title={Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity},
author={Zhou, Wenxuan and Held, David},
booktitle={Conference on Robot Learning (CoRL)},
year={2022}
}