Tensorflow implementation of Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matching, a paper at ICCV2021.
If you find this project useful in your research, please consider citing:
@inproceedings{2DProjectionMatching,
title={Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matching},
author={Chao Chen and Zhizhong Han and Yu-shen Liu and Matthias Zwicker},
booktitle={Proceedings of the IEEE International Conference on Computer Vision (ICCV)},
year={2021}
}
Our previous work published at ICML 2020 resolves the same problem from another perspective, please see here :
@inproceedings{handrwr2020,
author = {Zhizhong Han and Chao Chen and Yu-Shen Liu and Matthias Zwicker},
title = {{DRWR}: A Differentiable Renderer without Rendering for Unsupervised 3{D} Structure Learning from Silhouette Images},
booktitle = {International Conference on Machine Learning},
year = {2020},
}
The code is in Python 3.6.8. Create Python 3.6.8 environment:
conda create -n 2dpm python=3.6.8
conda activate 2dpm
Install dependencies:
pip install -r requirements.txt
we evaluate our method using ShapeNet v1 for all experiments.
The original ShapeNet has no corresponding point clouds and rendered images. Therefore, we need to preprocess 3D meshes to obtain point clouds and rendered images.
We provide the same point clouds and rendered images of 3 classes(chair, plane, and car) used in our paper as DPC, you can download them by the link, which contains gt/ and render/. the point clouds are only for test. You can also generate ground truth point clouds yourself as described here.
Firstly, put the gt/ folder and the render/ folder into the data/ folder.
Secondly, Using the original rendered images to generate silhouettes and 2D sampling points, and save them into TFrecords format (taking the plane(category ID 02691156) as an example):
cd data
./tf_records_generator.sh 02691156
A few hours later, you will see the tf_records/02691156_train.tf_records.
For convenience, we provide our generated TFrecords files of 3 classes(chair, plane, and car) in the link, which contains tf_records/. you can just put the tf_records/ folder into the data/ folder.
We also provide our pretrained model pretrained_model/ and generated shapes generated_shapes/ in the link. Put the pretrained_model/ into your checkpoint_dir.
To train our model, you can execute the following, taking the plane(category ID 02691156) as an example:
python 2Dpm/main/train_eval.py --gpu=0 --synth_set=02691156 --checkpoint_dir=./
All trained models will be saved in checkpoint_dir.
See the configurations in 2Dpm/resources/default_config.yaml for more details.
python 2Dpm/main/test.py --gpu=0 --synth_set=02691156 --checkpoint_dir=./ --test_step=100000
After the test, we save the quantification results in checkpoint_dir/chamfer_distance.txt. The generated 3D shapes are saved in checkpoint_dir/$vox_size/pred.
We thank DPC for their great works and repos.