Rina Wu, Tianqiang Zhu, Wanli Peng, Jinglue Hang, Yi Sun † ;
Dalian University of Technology
† corresponding author
IEEE Xplore
To assist or replace human beings in completing various tasks, research on the functional grasp synthesis of dexterous hands with high degree-of-freedom (DoF) is necessary and challenging. The dexterous functional grasp requires not only that the grasp is stable but more importantly facilitates the functional manipulation after grasping. Such work still relies on manual annotation when collecting data. To this end, we propose a category-level multi-fingered functional grasp transfer framework, in which we only need to label the hand-object contact relationship on functional parts of one object, and then transfer the contact information through the dense correspondence of functional parts between objects, so as to achieve the functional grasp synthesis for new objects based on the transferred hand-object contact information. We verify this method on three categories of representative objects through simulation experiments and achieve successful functional grasps by labeling only one instance in each category.
-
Ubuntu 18.04 or 20.04
-
Python 3.7
-
PyTorch 1.8.1
-
torchmeta 1.8.0
-
NVIDIA driver version >= 460.32
-
CUDA 10.2
- Download source meshes and grasp labels for knife, bottle categories from ShapeNetCore dataset or Baidu:-ecgf.
- Please download the trained model of this project from Baidu:-ecgf. And arrange the files as follows:
|-- FGtrans
|-- DIF
|-- models
|-- knife
|-- bottle
|-- checkpoints
|--model.pth
- Copy all folders in '02876657'(bottle) to datasets, and file '03624134'(knife) as above. And arrange the files as follows:
|-- FGtrans
|-- datasets
|-- obj
|-- knife
|-- bottle
|-- files
(*Here we take **bottle** as an example.*)
```shell
# Preprocess data, run:
python processdata/preprocess_ShapeNet.py --category bottle
# Generate sdf values, run:
python processdata/generate_sdf.py --category bottle --mode train
python processdata/generate_sdf.py --category bottle --mode eval
```
-
To train DIF, run the following commands.
python DIF/train.py \ --config DIF/configs/train/bottle.yml
-
To transfer touch code, run the following commands. Please modify the path of the split in the config file, such as "split/eval/batch. txt" to "split/train/batch. txt" to generate transfer data for all objects.
python DIF/evaluate.py \ --config DIF/configs/eval/bottle.yml
-
Convert the touch code to the original data.
python processdata/change_to_real_obj.py --category bottle --mode train python processdata/change_to_real_obj.py --category bottle --mode eval python processdata/remove_redundancy.py --category bottle
This repo is based on DIF-Net. Many thanks for their excellent works.
@article{wu2023functional,
title={Functional Grasp Transfer Across a Category of Objects From Only one Labeled Instance},
author={Wu, Rina and Zhu, Tianqiang and Peng, Wanli and Hang, Jinglue and Sun, Yi},
journal={IEEE Robotics and Automation Letters},
volume={8},
number={5},
pages={2748--2755},
year={2023},
publisher={IEEE}
}
Our code is released under MIT License.