This is the original PyTorch implementation of PointCAT: Cross-Attention Transformer for point cloud.
Paper link: https://arxiv.org/pdf/2304.03012.pdf.
The latest codes are tested on Windows 10, CUDA11.3, PyTorch 1.11.0 and Python 3.9:
conda install pytorch==1.11.0 torchvision==0.12.0 torchaudio==0.11.0 cudatoolkit=11.3 -c pytorchpointnet2_ops_lib:
pip install pointnet2_ops_lib/.einops==0.6.0
h5py==3.6.0
matplotlib==3.5.2
numpy==1.21.5
open3d==0.15.1
pandas==1.4.2
pointnet2_ops==3.0.0
pyntcloud==0.3.1
scikit_learn==1.2.2
scipy==1.8.0
thop==0.1.1.post2209072238
timm==0.6.7
tqdm==4.64.0
Download ModelNet40 dataset here and save in classification_ModelNet40\utils\data.
# Classification on ModelNet40
cd ./classification_ModelNet40
# Train
python train.py --batch_size 16 --use_sgd --learning_rate 0.01
# Test
python test.py --batch_size 16 --checkpoint your_chekpoint_root Download our best model in google drive here and save in ./classification_ModenNet40/checkpoints/best_checkpoints.
python test.py --checkpoint checkpoints/best_checkpoints ## Test result: 93.5% OA & 90.9% mAcc| Model | Inputs | Over All Accuracy (%) | Mean Accuracy (%) | Vote |
|---|---|---|---|---|
| PointCAT | 1024 points | 93.5% | 90.9% | × |
Download alignment ShapeNetPart here and save in part_segmentation/data/shapenetcore_partanno_segmentation_benchmark_v0_normal/.
# Part Segmentation on ShapeNetPart
cd ./part_segmentation
# Train
python train.py --exp_name "your_exp_name" --lr 0.003 --batch_size 16 --epochs 250
# Test
python train.py --exp_name "your_exp_name" --eval| Cls. mIoU | Inst. mIoU | aero | bag | cap | car | chair | earphone | guitar |
|---|---|---|---|---|---|---|---|---|
| 84.4 | 86.0 | 83.0 | 83.8 | 90.1 | 79.8 | 90.2 | 83.4 | 91.8 |
| knife | lamp | laptop | motorbike | mug | pistol | rocket | skateboard | table |
| 87.8 | 82.5 | 95.9 | 76.1 | 95.4 | 84.9 | 68.5 | 83.1 | 84.3 |
# Visualization of Part Segmentation Results
cd ./part_segmentation
img_root = './data/shapenetcore_partanno_segmentation_benchmark_v0_normal'
model_root = './checkpoints/your_experiment_root' # dir to your checkpoints
target_root = './results/your_result_root' # dir to save the result
# Visualization
python vis.py # 
Download 3D indoor parsing dataset (S3DIS) here and save in data/s3dis/Stanford3dDataset_v1.2_Aligned_Version/.
cd data_utils
python collect_indoor3d_data.py
Processed data will save in data/stanford_indoor3d/. The data structure be like:
├─stanford_indoor3d
| ├─Area_1_conferenceRoom_1.npy
| ├─Area_1_conferenceRoom_2.npy
| ├─Area_1_copyRoom_1.npy
| ├─Area_1_hallway_1.npy
| ├─Area_1_hallway_2.npy
| ├─Area_1_hallway_3.npy
...............
# Semantic Segmentation on S3DIS
cd ./seg_segmentation
# Train
python train.py --learning_rate 0.001 --optimizer AdamW --test_area 5
# Test
python test_semseg.py --log_dir your_expriment_root --visiual --test_area 5Visualization results will save in log/sem_seg/your_expriment_root/visual/ and you can transform these .obj files to .txt by these command:
cd ./seg_segmentation/data
# Vis
python input_vis.py
If you find this repo useful in your research, please consider citing it and our other works:
@article{Pytorch_Pointnet_Pointnet2,
Author = {Xu Yan},
Title = {Pointnet/Pointnet++ Pytorch},
Journal = {https://github.com/yanx27/Pointnet_Pointnet2_pytorch},
Year = {2019}
}
@misc{guo2020pct,
title={PCT: Point Cloud Transformer},
author={Meng-Hao Guo and Jun-Xiong Cai and Zheng-Ning Liu and Tai-Jiang Mu and Ralph R. Martin and Shi-Min Hu},
year={2020},
eprint={2012.09688},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{yang2023pointcat,
title={PointCAT: Cross-Attention Transformer for point cloud},
author={Xincheng Yang and Mingze Jin and Weiji He and Qian Chen},
year={2023},
eprint={2304.03012},
archivePrefix={arXiv},
primaryClass={cs.CV}
}