This ROS package implements PSSNet, a neural network that takes in a grid of visible occupied voxels from a single view and outputs a grid of the estimated 3D voxels, thus "completing the shape". Running multiple inference passes with the same input will generate different, yet plausible completions. Depending on the ambiguity of the input the completions may all be quite similar, or vary noticably. For Details, see the paper: Diverse Plausible Shape Completions from Ambiguous Depth Images (link coming soon)
To view completions of the test dataset.
roslaunch shape_completion_visualization shape_completion.launch
cd shape_completion_visualization/scripts
,./data_publisher.py --trial PSSNet/September_10_21-15-32_f87bdf38d4
In RViz you can then cycle through the various shapes and increasing angles. Change the --trial
arguments to explore the different networks and the YCB vs shapenet datasets.
- Install
ROS
kinetic or melodic - Clone this package into your ros workspace
- Clone the RViz plugins for text seletion and Voxelgrids into your ros workspace
- Install the python dependencies
pip install -r requirements.txt
.- install the
CUDA
drivers necessary for tensorflow - see below if there are issues with `python-pcl
- install the
- Build and resource the catkin workspace
- Download the pretrained model and preprocessed data:
./scripts/download_pretrained.py
You do not have to use the pretrained models and preprocessed datasets. You can do all or any part of the following:
- Download shapenet or ycb and run
augment_{shapenet, ycb}.py
- Compute the plausible set:
{shapenet, ycb}_plausibilities.py
- train a network
train.py --group {PSSNet, or other}
. If using PSSNet, firsttrain_flow.py
- evaluate a network compared to the plausible set. Edit and run
evaluate.py
shape_completion_training
: Generating, training, evaluating the shape completion model and baselinesshape_completion_visualization
: Scripts for viewing shape datasets and completions in RViz
###The code is developed and tested on
CUDA
10.2cuDNN
7.6Python
2.7.12 and 3.8.5TensorFlow
2.1ROS
kinetic or melodic- For visualization in RViz: https://github.com/bsaund/rviz_text_selection_panel and https://github.com/bsaund/rviz_voxelgrid_visuals
The pcl version you need to install depends on OS and python version
Python 2, Ubuntu 18.04: pip install python-pcl
I have sometimes found issues depending on pcl
verion. see: strawlab/python-pcl#317
Python 3, Ubuntu 20.04: sudo apt install python3-pcl