SEPT

Source code for paper Wireless Point Cloud Transmission, https://arxiv.org/abs/2306.08730

Motivation

There have already been considerable amount of works for semantic communications concerning images, speech and videos, while the transmission of point cloud is relatively less explored. Engineerers in the past would first compress the point cloud into bits using point cloud compression algorithm (e.g., G-PCC & V-PCC) and the use channel codes to protect them. This would cause cliff and leveling effects. This work serves as one of the pioneering works introducing a deep joint source channel coding (DeepJSCC) approach to avoid the cliff effect and achieving impressive reconstruction performance.

Install

Pytorch3d (please closely follow the instructions detailed in pytorch3d).
h5py (for dataset) & open3d (for visualization):

 pip install h5py & open3d

dataset: shapenetv2, please closely follow the precedule described in the link. The number of points is sampled to 2048 via fps (furthest point sampling) algorithm.

Usage

Train:

Change the hyper parameters, e.g., snr and bottleneck_size, then type

python train_pctrans.py

Test:

We provide checkpoint for the setting snr = 10, reconstructed_points = 2048 and bottleneck_size = 300, available here. Place it in the folder 'log/300_2048_snr10/checkpoints' Change the parameters and type for evaluation.

python test_pctrans.py

Results

Qualitive results

We show in the left figure that the proposed scheme can outperform the DPCC delivered under the achievable rate of finite block length (finite block length channel coding). We also show with a setting more favorable to the conventional GPCC algorithm, we can still outperform GPCC + finite block length achievable rate. The figure in the right shows SEPT avoids cliff effect.

Visulization

Left: original image; middle: snr = 5 dB; Right: snr = 0 dB.

Limitations

The proposed SEPT is more beneficial for handling small point clouds (e.g., in the shapenetv2 with 2k points), it tends to be less effective when handling that in depoco for dense point cloud map with at least 30k points. It's definitely a future direction.

Citation

If you find the code helpful in your research or work, please consider cite:

@misc{sept,
      title={Wireless Point Cloud Transmission}, 
      author={Chenghong Bian and Yulin Shao and Deniz Gunduz},
      year={2023},
      eprint={2306.08730},
      archivePrefix={arXiv},
      primaryClass={eess.SP}
}

Acknowledgements

We sincerely thanks the authors of DPCC for insightful discussion. We also recommend the readers to other related repos for more info point transformer, depoco, finite block length channel coding.

aprilbian/SEPT