/3D-object-localization

Point Cloud object localization under the guideline of natural language description

Primary LanguagePython

3D Object Localization

Dependencies

The code is tested on Ubuntu 18.04.

  • PyTorch v1.1, CUDA 10.0 and cuDNN v7.4
  • tensorflow or tensorflow-gpu v1.14.0 (for the tensorboard)

Dataset

The model in this repository is from torchvision which contains pretrained ResNet50-Backbone and then fine-tuned on ScanNet dataset. In order to obtain the data, please follow this README.

Data preparation

After downloading the dataset from ScanNet, place them under the /data folder. You will have 3 folders:

  • frames_square for rgb images, depth images and semantics segmentation labels(which is not needed here)
  • scannet_frame_labels contains pixelwise instance segmentation labels
  • scannet_frame_bbox contains labels for bounding box, object_id, semantic_label_id

So your /data folder will look like this:

  • data
    • frames_square
      • scene0000_01
        • color
        • depth
    • scannet_frame_labels
      • scene0000_01
        • instance-filt
          • 0.png
          • 1.png
    • scannet_frame_bbox
      • scene0000_01
        • 0.p
        • 20.p

Run batch_load_scannet_data.pyto organize the data so that it will be easy to use our dataloader to train the model.

Tensorboard

By default, the log for tensorboard is stored in /train_log under the root folder of this repository. Simply cd to this repository and run:

tensorboard --logdir=./train_log

Troubleshoot

You might need to conda install -c anaconda future in your conda environment.

Development

  • Plot function for loss history
  • Add other backbone network
  • Add Adam optimizer