This repository includes code used in our work on robust grasp planning over uncertain shape completions. More specifically, it includes code to train and test our shape completion network and to do the shape reconstruction experiment.
Authors: Jens Lundell
Maintainer: Jens Lundell, jens.lundell@aalto.fi
Affiliation: Intelligent Robotics Lab, Aalto University
The code was developed for python2.7 and Ubuntu 18.04.
ROS Melodic if and only if you want to integrate shape completion into, e.g., a grasping pipeline.
If you are not going to run this code as a part of a ROS setup then do the following:
python setup.py install --user
If you want to do online shape completion of point-clouds acquired from a camera you need to do the following:
Clone or download the project from Github:
cd <PATH_TO_YOUR_CATKIN_WORKSPACE>/src
git clone git@github.com:aalto-intelligent-robotics/shape_completion_network.git
Compile the ROS workspace
cd <PATH_TO_YOUR_CATKIN_WORKSPACE>
catkin_make
If you neither want to train a new network nor run the shape reconstruction experiment then skip this section.
To download the ground truth meshes, and the corresponding training data in the form of voxel grids either
- go to the following link and download training_data.zip and ground_truth_meshes.zip
- or alternatively run the following bash script
cd datasets
bash download_data.sh
Next, to parse the data into a training and holdout set run
python build_dataset.py [-h] [--dataset-type [{train_test,debug}]]
[--path-to-dataset PATH_TO_DATASET]
[--dataset-seed DATASET_SEED]
[--use-default-split USE_DEFAULT_SPLIT]
If you use this training data then please also cite the following work.
If you want to use one of the pre-trained models mentioned in the paper then either:
- go to the following link and download the models,
- or alternatively run the following bash script
cd networks
bash download_networks.sh
shape_completion_main.py [-h]
[--network-model [{mc_dropout,stein,varley}]]
[--use-batch-norm USE_BATCH_NORM]
[--mode [{train,test}]] [--lr LR]
[--train-in-parallel TRAIN_IN_PARALLEL]
[--latent-dim LATENT_DIM]
[--num-epochs NUM_EPOCHS]
[--batch-size BATCH_SIZE]
[--num-workers NUM_WORKERS]
[--test-cases [{train_models_train_views,train_models_holdout_views,holdout_models_holdout_views,all}]]
[--num-particles NUM_PARTICLES]
[--num-test-samples NUM_TEST_SAMPLES]
[--dropout-rate DROPOUT_RATE]
[--checkpoint-interval CHECKPOINT_INTERVAL]
[--num-objects-to-test NUM_OBJECTS_TO_TEST]
[--save-location SAVE_LOCATION]
[--regularization REGULARIZATION]
[--use-cuda USE_CUDA]
[--net-recover NET_RECOVER]
[--net-recover-name NET_RECOVER_NAME]
[--net-recover-epoch NET_RECOVER_EPOCH]
[--use-skip-connections USE_SKIP_CONNECTIONS]
[--debug DEBUG] [--save-samples SAVE_SAMPLES]
[--save-mesh SAVE_MESH]
[--save-voxel-grid SAVE_VOXEL_GRID]
shape_completion_main.py --mode train [-h]
[--network-model [{mc_dropout,stein,varley}]]
[--use-batch-norm USE_BATCH_NORM]
[--lr LR]
[--train-in-parallel TRAIN_IN_PARALLEL]
[--latent-dim LATENT_DIM]
[--num-epochs NUM_EPOCHS]
[--batch-size BATCH_SIZE]
[--num-workers NUM_WORKERS]
[--num-particles NUM_PARTICLES]
[--dropout-rate DROPOUT_RATE]
[--checkpoint-interval CHECKPOINT_INTERVAL]
[--save-location SAVE_LOCATION]
[--regularization REGULARIZATION]
[--use-cuda USE_CUDA]
[--net-recover NET_RECOVER]
[--net-recover-name NET_RECOVER_NAME]
[--net-recover-epoch NET_RECOVER_EPOCH]
[--use-skip-connections USE_SKIP_CONNECTIONS]
[--debug DEBUG]
When testing the network, the network first reads in point-clouds from the test-set, converts these into voxel grids, shape completes the voxel grid, ,, combines the original point-cloud with the shape completed voxel grid, and finally runs marching cube algorithm to get a mesh.
For testing a network run
shape_completion_main.py --mode test --batch-size 1 --num-workers 1
[-h]
[--network-model [{mc_dropout,stein,varley}]]
[--use-batch-norm USE_BATCH_NORM]
[--latent-dim LATENT_DIM]
[--test-cases [{train_models_train_views,train_models_holdout_views,holdout_models_holdout_views,all}]]
[--num-test-samples NUM_TEST_SAMPLES]
[--dropout-rate DROPOUT_RATE]
[--num-objects-to-test NUM_OBJECTS_TO_TEST]
[--save-location SAVE_LOCATION]
[--use-cuda USE_CUDA]
[--net-recover NET_RECOVER]
[--net-recover-name NET_RECOVER_NAME]
[--use-skip-connections USE_SKIP_CONNECTIONS]
[--save-samples SAVE_SAMPLES]
[--save-mesh SAVE_MESH]
[--save-voxel-grid SAVE_VOshape completed meshes b
In this section, we detail how you can run the shape reconstruction experiment as explained in Section IV-B in our paper. However, before you can run the experiment you need some shape completed meshes. You can either generate new ones by following this or, alternatively, download some already shape completed meshes here.
After you have procured the meshes, run
shape_reconstruction_main.py [-h]
[--completion-method [{mc_dropout,stein,varley}]]
[--save-dir SAVE_DIR]
[--ground-truth-data-dir GROUND_TRUTH_DATA_DIR]
[--shape-completed-data-dir SHAPE_COMPLETED_DATA_DIR]
This repository contains code corresponding to our work on robust grasp planning over uncertain shape completions. If you use the code for your research, please cite
@article{lundell2019robust,
title={Robust Grasp Planning Over Uncertain Shape Completions},
author={Lundell, Jens and Verdoja, Francesco and Kyrki, Ville},
journal={arXiv preprint arXiv:1903.00645},
year={2019}
}
The original training data was not created by us but by Jacob Varley et al. and we only have the authors approval to redistribute. Thus, if you use the training data in your research project then please also cite the original work
@inproceedings{varley2017shape,
title={Shape completion enabled robotic grasping},
author={Varley, Jacob and DeChant, Chad and Richardson, Adam and Ruales, Joaqu{\'\i}n and Allen, Peter},
booktitle={2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
year={2017},
organization={IEEE}
}
This project is licensed under the MIT License - see the LICENSE file for details