Paper can be seen from this link.
If you use our work in your research, please cite it like this:
@inproceedings{learntodig2022,
title={Learning to Pick by Digging: Data-Driven Dig-Grasping for BinPicking from Clutter},
author={Zhao, Chao and Tong, Zhekai and Rojas, Juan and Seo, Jungwon},
booktitle={2022 IEEE International Conference on Robotics and Automation (ICRA)},
year={2022},
organization={IEEE}
}
In this work, we propose a learning-based method for bin picking, which aims at singulating and simultaneously picking the objects one by one from a random clutter. It illustrates a way of picking through a physical interaction between the robot's gripper and object clutter, which is essential to successful singulation (and subsequent picking), realized as a digging operation along a straight line. A gripper designed for this technique is capable of changing relative digit lengths such that the object being digged will not collide with the other finger. This repository provides the PyTorch implementation for training and testing this interactive picking strategy.
The following figure shows an overall process. Given a depth image, the robot learns a way of interaction (encoded in the score map) with the target object to pick up. The object (blue block) is rotated by the finger pushing it down to the clutter.
Video demonstration:
Full Video can be seen from this link.
- Universal Robot UR10
- Robotiq 140mm Adaptive parallel-jaw gripper
- RealSense Camera L515
- Extendable Finger for realizing the adjustable finger length. The CAD model can be found here.
The code is built with Python 3.6. Dependencies are listed in [requirements.yaml] and can be installed via Anaconda by running:
conda env create -n learn_dig -f requirements.yaml
This demo runs our trained model in simulation, which illustrates the bin picking of domino blocks from a cluttered bin.
Instruction
- Download this repository:
https://github.com/HKUST-RML/Learning-to-Grasp-by-Digging_v2.git
- Run the script (the trained model will be downloaded automatically):
python demo.py
The entire training process is repeated 7 times in a self-supervised manner. At each time, we recollect a dataset that contains 5000 scenes (bins of objects).
We provide here the dataset already collected in our last training process.
You can train the model with this provided dataset by running:
python train_last_model.py
If you want to create your own dataset (currently the trained objects are domino blocks, coin-shaped short cylinder, and short triangular prisms), and start training the models from scratch, please run the following code:
python trainer.py
We provide a testing script to evaluate our trained model in simulation. The following code runs the test on three trained objects, and report the average grasp success rates.
python test_in_sim.py
Here we provide the steps to test our method on a real robot.
Robot control
Robot is controlled via this python software.
Camera setup
To deploy RealSense L515 camera,
- Download and install the librealsense SDK 2.0
- Our camera setting can be found in
real/640X480_L_short_default.json
Start testing
Then run the following code to start testing:
cd real
python test_in_real.py
For any technical issues, please contact: Chao Zhao (czhaobb@connect.ust.hk), Zhekai Tong (ztong@connect.ust.hk).