Implicit Functions in Feature Space for Shape Reconstruction and Completion
Julian Chibane, Thiemo Alldieck, Gerard Pons-Moll
Paper - Supplementaty - Project Website - Arxiv - Video - Slides - Published in CVPR 2020.
@inproceedings{chibane20ifnet,
title = {Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion},
author = {Chibane, Julian and Alldieck, Thiemo and Pons-Moll, Gerard},
booktitle = {{IEEE} Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {jun},
organization = {{IEEE}},
year = {2020},
}
A linux system with cuda 9.0 is required for the project.
The if-net_env.yml file contains all necessary python dependencies for the project.
To conveniently install them automatically with anaconda you can use:
conda env create -f if-net_env.yml
conda activate if-net
Please clone the repository and navigate into it in your terminal, its location is assumed for all subsequent commands.
This project uses libraries for Occupancy Networks by [Mescheder et. al. CVPR'19] and the ShapeNet data preprocessed for DISN by [Xu et. al. NeurIPS'19], please also cite them if you use our code.
Install the needed libraries with:
cd data_processing/libmesh/
python setup.py build_ext --inplace
cd ../libvoxelize/
python setup.py build_ext --inplace
cd ../..
Download the ShapeNet data preprocessed by [Xu et. al. NeurIPS'19] from here
into the shapenet folder.
Now extract the files into shapenet\data with:
ls shapenet/*.tar.gz |xargs -n1 -i tar -xf {} -C shapenet/data/
Next, the inputs and training point samples for IF-Nets are created. The following three commands can be run in parallel on multiple machines to significantly increase speed. First, the data is converted to the .off-format and scaled using
python data_processing/convert_to_scaled_off.py
The input data for Voxel Super-Resolution of voxels is created with
python data_processing/voxelize.py -res 32
using -res 32 for 323 and -res 128 for 1283 resolution.
The input data for Point Cloud Completion is created with
python data_processing/voxelized_pointcloud_sampling.py -res 128 -num_points 300
using -num_points 300 for point clouds with 300 points and -num_points 3000 for 3000 points.
Training input points and the corresponding ground truth occupancy values are generated with
python data_processing/boundary_sampling.py -sigma 0.1
python data_processing/boundary_sampling.py -sigma 0.01
where -sigma specifies the standard deviation of the normally distributed displacements added onto surface samples.
In order to remove meshes that could not be preprocessed (should not be more than around 15 meshes) you should run
python data_processing/filter_corrupted.py -file 'voxelization_32.npy' -delete
The input data can be visualized by converting them to .off-format using
python data_processing/create_voxel_off.py -res 32
for voxel input and
python data_processing/create_pc_off.py -res 128 -num_points 300
where -res and -num_points matches the values from the previous steps.
The training of IF-Nets is started running
python train.py -std_dev 0.1 0.01 -res 32 -m ShapeNet32Vox -batch_size 6
where -std_dev indicates the sigmas to use, -res the input resolution (323 or 1283), -m the IF-Net model setup
- ShapeNet32Vox for 323 voxel Super-Resolution experiment
- ShapeNet128Vox for 1283 voxel Super-Resolution experiment
- ShapeNetPoints for Point Cloud Completion experiments
- SVR for 3D Single-View Reconstruction
and -batch_size the number of different meshes inputted in a batch, each with 50.000 point samples (=6 for small GPU's). Consider using the highest possible batch_size in order to speed up training.
In the experiments/ folder you can find an experiment folder containing the model checkpoints, the checkpoint of validation minimum, and a folder containing a tensorboard summary, which can be started at with
tensorboard --logdir experiments/YOUR_EXPERIMENT/summary/ --host 0.0.0.0
The command
python generate.py -std_dev 0.1 0.01 -res 32 -m ShapeNet32Vox -checkpoint 10 -batch_points 400000
generates the reconstructions of the, during training unseen, test examples from ShapeNet into the folder
experiments/YOUR_EXPERIMENT/evaluation_CHECKPOINT_@256/generation.
With -checkpoint you can choose the IF-Net model checkpoint. Use the model with minimum validation error for this,
-batch_points indicates the number of points that fit into GPU memory at once (400k for small GPU's), the other parameters are set as during training.
The generation script can be run on multiple machines in parallel in order to increase generation speed significantly. Also, consider using the maximal batch size possible for your GPU.
Please run
python data_processing/evaluate.py -reconst -generation_path experiments/iVoxels_dist-0.5_0.5_sigmas-0.1_0.01_v32_mShapeNet32Vox/evaluation_10_@256/generation/
to evaluate each reconstruction, where -generation_path is the path to the reconstructed objects generated in the previous step.
The above evaluation script can be run on multiple machines in parallel in order to increase generation speed significantly.
Then run
python data_processing/evaluate.py -voxels -res 32
to evaluate the quality of the input. For voxel girds use '-voxels' with '-res' to specify the input resolution and for point clouds use '-pc' with '-points' to specify the number of points.
The quantitative evaluation of all reconstructions and inputs are gathered and put into experiment/YOUR_EXPERIMENT/evaluation_CHECKPOINT_@256 using
python data_processing/evaluate_gather.py -voxel_input -res 32 -generation_path experiments/iVoxels_dist-0.5_0.5_sigmas-0.1_0.01_v32_mShapeNet32Vox/evaluation_10_@256/generation/
where you should use -voxel_input for Voxel Super-Resolution experiments, with -res specifying the input resolution or -pc_input for Point Cloud Completion, with -points specifying the number of points used.
For questions and comments regarding the code please contact Julian Chibane via mail. (See Paper)