CurveCloudNet: Processing Point Clouds with 1D Structure

Installation

Python Environment

For quick installation, install anaconda and run bash setup.sh ANACONDA_PATH ENV_NAME. For example, bash setup.sh /home/colton/anaconda3 curvenet.

This repo was tested on Ubuntu 20.04 with Cuda 11.7 and PyTorch 2.0. The external package FRNN does NOT work with CUDA 12!

Note that we use Weights and Biases for visualizations and run metrics. In order to access any program outputs, please create a Weights and Biases account.

Data

nuScenes Dataset

Please go to the official website and make an account to download the nuScenes dataset. For this repo, you will need the nuScenes full dataset (v1.0) and the nuScenes Lidarseg extension. Follow instructions to obtain a directory with the following structure, and place the directory under the ./data directory named as nuscenes:

THIS_DIR
   │── data
       │── nuscenes
            │── samples
            │── sweeps
            │── lidarseg
            ...
            │── v1.0-trainval 
            │── v1.0-test

Kortx Dataset

Download and unzip the Kortx dataset from this url. The following structure is expected (where each file is necessary):

kortx-processed
│── data_2048_2048_0.125_grid_val.pth
│── data_2048_summer_robotics_test.pth
│── data_2048_2048_0.125_grid_train.pth

Similar to the nuscenes dataset, place the directory in the ./data folder.

ShapeNet Laser Scanned Dataset

Download and unzip the ShapeNet dataset from this url. The following structure is expected (where each file is necessary):

shapenet-processed
│── data_2048_2048_0.25_*.pth

Similar to the other datasets, place the directory in the ./data folder.

Audi A2D2 Dataset

Download and unzip the already-preprocessed A2D2 dataset from this url. The following structure is expected

A2D2
│── Segmentation
    │── semantic_train
    │── semantic_val
    │── semantic_test
    │── cam_lidars.json
    │── camera_lidar_semantic_instance
    ...

Similar to the other datasets, place the directory in the ./data folder.

Running Code - Evaluation

All code is executed via the src/main.py file. Use help for config options.

PYTHONPATH=. python src/main.py --help

Downloading Weights

You can download all pretrained model weights here. After downloading, place the weights folder as follows:

THIS_DIR
   │── models
       │── curvecloudnet
            │── audi.pth
            │── kitti.pth
            ...
            │── shapeneg-seg-random.pth

NOTE: The checkpoints for the Audi Segmentation, ShapeNet Segmentation, and ShapeNet Classification were originally trained and evaluated on an older version of the code. For unknown reasons, these checkpoints are reporting slightly lower accuracies in this version. Please be patient as we re-train the models.

Evaluation Commands

You can use the following commands evaluate CurveCloudNet on each dataset.

nuScenes

PYTHONPATH=. python src/main.py --config ./configs/curvecloudnet-eval/nuscenes-curvecloudnet.yaml

Kitti

PYTHONPATH=. python src/main.py --config ./configs/curvecloudnet-eval/kitti-curvecloudnet.yaml

Audi

PYTHONPATH=. python src/main.py --config ./configs/curvecloudnet-eval/audi-curvecloudnet.yaml

Kortx

PYTHONPATH=. python src/main.py --config ./configs/curvecloudnet-eval/kortx-testsplit-curvecloudnet.yaml

ShapeNet Segmentation (grid sampling)

PYTHONPATH=. python src/main.py --config ./configs/curvecloudnet-eval/shapenet-seg-curvecloudnet.yaml

ShapeNet Classification (random sampling)

PYTHONPATH=. python src/main.py --config ./configs/curvecloudnet-eval/shapenet-class-curvecloudnet.yaml

Additional Useful Flags

The following flags are useful for evaluating pretrained models:

--only_viz=True: the model renders visualizations and then exits
--weights=<PATH_TO_PRETRAINED_WEIGHTS>: specify model model weights to load

Running Code - Training

Training Command

To train CurveCloudNet, simply run the desired training configuration:

PYTHONPATH=. python src/main.py --config ./configs/curvecloudnet-train/nuscenes-curvecloudnet.yaml

Other Config Parameters in Training

Coming soon!

coltonstearns/curvecloudnet