/splatnet

SPLATNet: Sparse Lattice Networks for Point Cloud Processing (CVPR2018)

Primary LanguagePythonOtherNOASSERTION

SPLATNet: Sparse Lattice Networks for Point Cloud Processing (CVPR2018)

License

Copyright (C) 2018 NVIDIA Corporation. All rights reserved. Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).

Paper

arXiv

@inproceedings{su18splatnet,
  author={Su, Hang and Jampani, Varun and Sun, Deqing and Maji, Subhransu and Kalogerakis, Evangelos and Yang, Ming-Hsuan and Kautz, Jan},
  title     = {{SPLATN}et: Sparse Lattice Networks for Point Cloud Processing},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages     = {2530--2539},
  year      = {2018}
}

Usage

  1. Install Caffe and bilateralNN

    Note that our code uses Python3.

    • Please follow the instructions on the bilateralNN repo.
    • A step-by-step installation guide for Ubuntu 16.04 is provided in INSTALL.md.
    • Alternatively, you can install nvidia-docker and use this docker image: 
      docker pull suhangpro/caffe:bpcn
      You can also build your own image with this Dockerfile.

  2. Include the project to your python path so imports can be found, e.g.

    export PYTHONPATH=<PATH_TO_PROJECT_ROOT>:$PYTHONPATH

  3. Download and prepare data files under folder data/

    See instructions in data/README.md.

  4. Usage examples

    • 3D facade segmentation

      • test pre-trained model 
        cd exp/facade3d
./dl_model_facade3d.sh  # download pre-trained model
SKIP_TRAIN=1 ./train_test.sh
        Prediction is output at pred_test.ply, with evaluation results in test.log.
      • or, train and evaluate 
        cd exp/facade3d
./train_test.sh

    • ShapeNet Part segmentation

      • test pre-trained model 
        cd exp/shapenet3d
./dl_model_shapenet3d.sh  # download pre-trained model
./test_only.sh
        Predictions are under pred/, with evaluation results in test.log.
      • or, train and evaluate 
        cd exp/shapenet3d
./train_test.sh

    • Joint 2D-3D experiments

      (coming soon)

References

We make extensive use of bilateralNN, which is proposed in these publications:

  • V. Jampani, M. Kiefel and P. V. Gehler. Learning Sparse High-Dimensional Filters: Image Filtering, Dense CRFs and Bilateral Neural Networks. CVPR, 2016.
  • M.Kiefel, V. Jampani and P. V. Gehler. Permutohedral Lattice CNNs. ICLR Workshops, 2015.